2024-09-13T09:14:02.715 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T09:14:02.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:14:02.812 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901036' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: true rbd clone copy on read: true global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T09:14:02.812 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T09:14:02.813 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T09:14:02.813 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T09:14:02.815 INFO:teuthology.task.internal:Checking packages... 2024-09-13T09:14:02.835 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T09:14:02.835 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T09:14:02.835 INFO:teuthology.packaging:ref: None 2024-09-13T09:14:02.836 INFO:teuthology.packaging:tag: None 2024-09-13T09:14:02.836 INFO:teuthology.packaging:branch: reef 2024-09-13T09:14:02.836 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:14:02.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-09-13T09:14:03.052 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843-g2c01458a-1focal 2024-09-13T09:14:03.054 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T09:14:03.062 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T09:14:03.062 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T09:14:03.106 INFO:teuthology.task.internal:Saving configuration 2024-09-13T09:14:03.120 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T09:14:03.135 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T09:14:03.153 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 09:08:10.674732', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T09:14:03.175 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 09:08:10.676645', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T09:14:03.175 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T09:14:03.184 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T09:14:03.184 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T09:14:03.185 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T09:14:03.257 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffae441f5b0>, signals=[15]) 2024-09-13T09:14:03.257 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T09:14:03.275 INFO:teuthology.task.internal:Opening connections... 2024-09-13T09:14:03.276 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:14:03.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:14:03.369 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:14:03.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:14:03.467 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T09:14:03.478 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-09-13T09:14:03.485 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-09-13T09:14:03.485 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-09-13T09:14:03.533 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="Ubuntu" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:ID=ubuntu 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE=debian 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="20.04" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:14:03.534 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_CODENAME=focal 2024-09-13T09:14:03.535 INFO:teuthology.orchestra.run.smithi063.stdout:UBUNTU_CODENAME=focal 2024-09-13T09:14:03.535 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-09-13T09:14:03.564 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-13T09:14:03.571 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-13T09:14:03.571 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-13T09:14:03.577 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Ubuntu" 2024-09-13T09:14:03.577 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T09:14:03.577 INFO:teuthology.orchestra.run.smithi099.stdout:ID=ubuntu 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE=debian 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="20.04" 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_CODENAME=focal 2024-09-13T09:14:03.578 INFO:teuthology.orchestra.run.smithi099.stdout:UBUNTU_CODENAME=focal 2024-09-13T09:14:03.578 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-13T09:14:03.597 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T09:14:03.609 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T09:14:03.640 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T09:14:03.640 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:14:03.643 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:14:03.649 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T09:14:03.674 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T09:14:03.674 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:14:03.689 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:14:03.707 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T09:14:03.755 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T09:14:03.773 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T09:14:03.774 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T09:14:03.774 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:14:03.774 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:14:03.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:14:03.780 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-09-13T09:14:03.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:14:03.781 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-13T09:14:03.787 INFO:teuthology.orchestra.run.smithi063.stdout:5.4.0-124-generic 2024-09-13T09:14:03.787 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.4.0-124-generic 2024-09-13T09:14:03.787 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get clean 2024-09-13T09:14:03.788 INFO:teuthology.orchestra.run.smithi099.stdout:5.4.0-124-generic 2024-09-13T09:14:03.789 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.4.0-124-generic 2024-09-13T09:14:03.789 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get clean 2024-09-13T09:14:03.875 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-13T09:14:03.953 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-09-13T09:14:04.034 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T09:14:04.034 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T09:14:04.034 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T09:14:04.098 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T09:14:04.103 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T09:14:04.220 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T09:14:04.302 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T09:14:04.408 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T09:14:04.428 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T09:14:04.525 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T09:14:04.539 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T09:14:04.553 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T09:14:04.556 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T09:14:04.556 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T09:14:04.570 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T09:14:04.572 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T09:14:04.576 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T09:14:04.576 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T09:14:04.577 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T09:14:04.577 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T09:14:04.580 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T09:14:04.581 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T09:14:04.587 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T09:14:04.592 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T09:14:04.593 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T09:14:04.593 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T09:14:04.594 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T09:14:04.605 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T09:14:04.605 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T09:14:04.609 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T09:14:04.609 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T09:14:04.612 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T09:14:04.613 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T09:14:04.617 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T09:14:04.624 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T09:14:04.625 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T09:14:04.626 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T09:14:04.626 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T09:14:04.656 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T09:14:04.674 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T09:14:04.683 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T09:14:04.689 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T09:14:04.690 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T09:14:04.690 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T09:14:04.690 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T09:14:04.711 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T09:14:04.711 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T09:14:04.714 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T09:14:04.714 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T09:14:04.720 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-13T09:14:04.729 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T09:14:04.731 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T09:14:04.731 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T09:14:04.732 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T09:14:04.742 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T09:14:04.742 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T09:14:04.743 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T09:14:04.743 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T09:14:04.743 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T09:14:04.759 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T09:14:04.759 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T09:14:04.760 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T09:14:04.798 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T09:14:05.042 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T09:14:05.211 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T09:14:05.224 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T09:14:05.224 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T09:14:05.319 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T09:14:05.320 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T09:14:05.339 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T09:14:05.339 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-13T09:14:05.373 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T09:14:05.392 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T09:14:05.401 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T09:14:05.402 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T09:14:05.402 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T09:14:05.455 INFO:teuthology.orchestra.run.smithi063.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T09:14:05.456 INFO:teuthology.orchestra.run.smithi063.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T09:14:05.457 INFO:teuthology.orchestra.run.smithi063.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi063.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T09:14:05.537 INFO:teuthology.orchestra.run.smithi063.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T09:14:05.538 INFO:teuthology.orchestra.run.smithi063.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T09:14:05.539 INFO:teuthology.orchestra.run.smithi063.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T09:14:08.935 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 21.8 MB in 3s (7,227 kB/s) 2024-09-13T09:14:09.135 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 21.8 MB in 3s (7,057 kB/s) 2024-09-13T09:14:10.477 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-13T09:14:10.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:14:10.589 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-13T09:14:10.795 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-09-13T09:14:10.806 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-13T09:14:10.807 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-13T09:14:10.822 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:14:10.893 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-09-13T09:14:11.018 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2024-09-13T09:14:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T09:14:11.018 INFO:teuthology.orchestra.run.smithi099.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T09:14:11.019 INFO:teuthology.orchestra.run.smithi099.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T09:14:11.020 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2024-09-13T09:14:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T09:14:11.020 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2024-09-13T09:14:11.020 INFO:teuthology.orchestra.run.smithi099.stdout: thermald 2024-09-13T09:14:11.074 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2024-09-13T09:14:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T09:14:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T09:14:11.074 INFO:teuthology.orchestra.run.smithi099.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T09:14:11.075 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2024-09-13T09:14:11.076 INFO:teuthology.orchestra.run.smithi099.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:14:11.117 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-09-13T09:14:11.117 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-09-13T09:14:11.258 INFO:teuthology.orchestra.run.smithi099.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T09:14:11.259 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 77.2 MB of archives. 2024-09-13T09:14:11.259 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T09:14:11.259 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T09:14:11.329 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-09-13T09:14:11.329 INFO:teuthology.orchestra.run.smithi063.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T09:14:11.329 INFO:teuthology.orchestra.run.smithi063.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T09:14:11.329 INFO:teuthology.orchestra.run.smithi063.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T09:14:11.330 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-09-13T09:14:11.330 INFO:teuthology.orchestra.run.smithi063.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T09:14:11.330 INFO:teuthology.orchestra.run.smithi063.stdout:Recommended packages: 2024-09-13T09:14:11.330 INFO:teuthology.orchestra.run.smithi063.stdout: thermald 2024-09-13T09:14:11.384 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-09-13T09:14:11.384 INFO:teuthology.orchestra.run.smithi063.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T09:14:11.384 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T09:14:11.384 INFO:teuthology.orchestra.run.smithi063.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T09:14:11.386 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages will be upgraded: 2024-09-13T09:14:11.386 INFO:teuthology.orchestra.run.smithi063.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:14:11.440 INFO:teuthology.orchestra.run.smithi063.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T09:14:11.440 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 77.2 MB of archives. 2024-09-13T09:14:11.440 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T09:14:11.440 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T09:14:11.735 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T09:14:11.826 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T09:14:12.325 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T09:14:12.326 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T09:14:12.326 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T09:14:12.423 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T09:14:12.434 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T09:14:12.435 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T09:14:12.708 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T09:14:13.140 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 77.2 MB in 1s (74.1 MB/s) 2024-09-13T09:14:13.390 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T09:14:14.101 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T09:14:14.101 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T09:14:14.101 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T09:14:14.378 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T09:14:14.435 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T09:14:15.109 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 77.2 MB in 3s (23.0 MB/s) 2024-09-13T09:14:15.190 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T09:14:15.195 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T09:14:15.387 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:16.991 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T09:14:16.996 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:17.079 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:17.497 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T09:14:17.510 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:17.604 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:18.750 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T09:14:18.766 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:18.808 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:18.988 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T09:14:19.003 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:19.108 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:20.218 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T09:14:20.232 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:20.274 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:23.725 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T09:14:23.890 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T09:14:24.286 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T09:14:24.419 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T09:14:24.764 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T09:14:24.777 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T09:14:24.821 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T09:14:25.107 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T09:14:25.232 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T09:14:25.582 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T09:14:25.719 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T09:14:26.380 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T09:14:26.398 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T09:14:26.438 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T09:14:30.674 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T09:14:30.696 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:30.740 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:31.781 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T09:14:31.802 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T09:14:31.839 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:33.763 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T09:14:33.884 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T09:14:34.259 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:34.456 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T09:14:34.552 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T09:14:34.907 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:35.531 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:36.142 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:36.725 INFO:teuthology.orchestra.run.smithi063.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T09:14:36.725 INFO:teuthology.orchestra.run.smithi063.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T09:14:36.883 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:37.731 INFO:teuthology.orchestra.run.smithi099.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T09:14:37.732 INFO:teuthology.orchestra.run.smithi099.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T09:14:37.869 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:38.134 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T09:14:38.260 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T09:14:38.394 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:38.645 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T09:14:38.780 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T09:14:38.931 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:39.004 INFO:teuthology.orchestra.run.smithi063.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:14:39.005 INFO:teuthology.orchestra.run.smithi063.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T09:14:39.162 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T09:14:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T09:14:39.414 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:39.565 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T09:14:39.691 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T09:14:39.818 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T09:14:39.900 INFO:teuthology.orchestra.run.smithi099.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:14:39.900 INFO:teuthology.orchestra.run.smithi099.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T09:15:01.515 INFO:teuthology.orchestra.run.smithi099.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:15:01.515 INFO:teuthology.orchestra.run.smithi099.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:15:01.525 INFO:teuthology.orchestra.run.smithi099.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:15:01.541 INFO:teuthology.orchestra.run.smithi099.stdout:Generating grub configuration file ... 2024-09-13T09:15:01.879 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T09:15:01.914 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T09:15:02.437 INFO:teuthology.orchestra.run.smithi099.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T09:15:02.448 INFO:teuthology.orchestra.run.smithi099.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T09:15:02.753 INFO:teuthology.orchestra.run.smithi063.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:15:02.754 INFO:teuthology.orchestra.run.smithi063.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:15:02.771 INFO:teuthology.orchestra.run.smithi063.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:15:02.789 INFO:teuthology.orchestra.run.smithi063.stdout:Generating grub configuration file ... 2024-09-13T09:15:03.160 INFO:teuthology.orchestra.run.smithi063.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T09:15:03.180 INFO:teuthology.orchestra.run.smithi063.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T09:15:03.664 INFO:teuthology.orchestra.run.smithi063.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T09:15:03.676 INFO:teuthology.orchestra.run.smithi063.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T09:15:05.158 INFO:teuthology.orchestra.run.smithi099.stdout:done 2024-09-13T09:15:05.600 DEBUG:teuthology.orchestra.run.smithi099:> dpkg -s linux-image-generic 2024-09-13T09:15:05.648 INFO:teuthology.orchestra.run.smithi099.stdout:Package: linux-image-generic 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Status: install ok installed 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Priority: optional 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Section: kernel 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Installed-Size: 20 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: amd64 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Source: linux-meta 2024-09-13T09:15:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Version: 5.4.0.195.193 2024-09-13T09:15:05.650 INFO:teuthology.orchestra.run.smithi099.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T09:15:05.650 INFO:teuthology.orchestra.run.smithi099.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:15:05.650 INFO:teuthology.orchestra.run.smithi099.stdout:Recommends: thermald 2024-09-13T09:15:05.650 INFO:teuthology.orchestra.run.smithi099.stdout:Description: Generic Linux kernel image 2024-09-13T09:15:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:15:05.650 INFO:teuthology.orchestra.run.smithi099.stdout: available. 2024-09-13T09:15:05.650 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T09:15:05.650 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:15:05.651 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:15:05.651 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T09:15:05.651 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:15:05.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get clean 2024-09-13T09:15:05.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-13T09:15:05.868 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T09:15:05.874 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T09:15:05.891 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T09:15:06.004 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T09:15:06.462 INFO:teuthology.orchestra.run.smithi063.stdout:done 2024-09-13T09:15:06.965 DEBUG:teuthology.orchestra.run.smithi063:> dpkg -s linux-image-generic 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Package: linux-image-generic 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Status: install ok installed 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Priority: optional 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Section: kernel 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Installed-Size: 20 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: amd64 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Source: linux-meta 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Version: 5.4.0.195.193 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T09:15:07.007 INFO:teuthology.orchestra.run.smithi063.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:15:07.008 INFO:teuthology.orchestra.run.smithi063.stdout:Recommends: thermald 2024-09-13T09:15:07.008 INFO:teuthology.orchestra.run.smithi063.stdout:Description: Generic Linux kernel image 2024-09-13T09:15:07.008 INFO:teuthology.orchestra.run.smithi063.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:15:07.008 INFO:teuthology.orchestra.run.smithi063.stdout: available. 2024-09-13T09:15:07.008 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T09:15:07.008 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:15:07.008 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:15:07.008 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T09:15:07.009 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:15:07.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get clean 2024-09-13T09:15:07.103 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-09-13T09:15:07.238 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T09:15:07.238 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T09:15:07.244 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T09:15:07.262 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T09:15:07.737 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-13T09:15:07.765 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:15:07.839 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-13T09:15:08.056 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-13T09:15:08.057 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-13T09:15:08.286 INFO:teuthology.orchestra.run.smithi099.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T09:15:08.286 INFO:teuthology.orchestra.run.smithi099.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T09:15:08.288 DEBUG:teuthology.orchestra.run.smithi099:> dpkg -s linux-image-generic 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Package: linux-image-generic 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Status: install ok installed 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Priority: optional 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Section: kernel 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Installed-Size: 20 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:15:08.308 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: amd64 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout:Source: linux-meta 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout:Version: 5.4.0.195.193 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout:Recommends: thermald 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout:Description: Generic Linux kernel image 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:15:08.309 INFO:teuthology.orchestra.run.smithi099.stdout: available. 2024-09-13T09:15:08.310 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-13T09:15:08.315 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.aEWUOEYEll 2024-09-13T09:15:08.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub/grub.cfg /tmp/tmp.aEWUOEYEll 2024-09-13T09:15:08.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.aEWUOEYEll 2024-09-13T09:15:08.463 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.aEWUOEYEll is 9KB 2024-09-13T09:15:08.478 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.aEWUOEYEll 2024-09-13T09:15:08.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:15:08.541 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:15:08.541 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:15:08.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:15:08.575 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T09:15:08.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo update-grub 2024-09-13T09:15:09.135 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-09-13T09:15:09.163 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:15:09.234 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-09-13T09:15:09.458 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-09-13T09:15:09.458 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-09-13T09:15:09.707 INFO:teuthology.orchestra.run.smithi063.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T09:15:09.707 INFO:teuthology.orchestra.run.smithi063.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T09:15:09.709 DEBUG:teuthology.orchestra.run.smithi063:> dpkg -s linux-image-generic 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Package: linux-image-generic 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Status: install ok installed 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Priority: optional 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Section: kernel 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Installed-Size: 20 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: amd64 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Source: linux-meta 2024-09-13T09:15:09.728 INFO:teuthology.orchestra.run.smithi063.stdout:Version: 5.4.0.195.193 2024-09-13T09:15:09.729 INFO:teuthology.orchestra.run.smithi063.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T09:15:09.729 INFO:teuthology.orchestra.run.smithi063.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:15:09.729 INFO:teuthology.orchestra.run.smithi063.stdout:Recommends: thermald 2024-09-13T09:15:09.729 INFO:teuthology.orchestra.run.smithi063.stdout:Description: Generic Linux kernel image 2024-09-13T09:15:09.729 INFO:teuthology.orchestra.run.smithi063.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:15:09.729 INFO:teuthology.orchestra.run.smithi063.stdout: available. 2024-09-13T09:15:09.729 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-09-13T09:15:09.735 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.coXB0rqg8e 2024-09-13T09:15:09.736 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub/grub.cfg /tmp/tmp.coXB0rqg8e 2024-09-13T09:15:09.746 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.coXB0rqg8e 2024-09-13T09:15:09.829 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.coXB0rqg8e is 9KB 2024-09-13T09:15:09.843 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.coXB0rqg8e 2024-09-13T09:15:09.849 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:15:09.859 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:15:09.859 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:15:09.859 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:15:09.902 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T09:15:09.902 DEBUG:teuthology.orchestra.run.smithi063:> sudo update-grub 2024-09-13T09:15:10.015 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:15:10.022 INFO:teuthology.orchestra.run.smithi099.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:15:10.030 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-13T09:15:10.328 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T09:15:10.340 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T09:15:10.695 INFO:teuthology.orchestra.run.smithi099.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T09:15:10.707 INFO:teuthology.orchestra.run.smithi099.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T09:15:11.329 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:15:11.337 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:15:11.388 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-09-13T09:15:11.682 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T09:15:11.693 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T09:15:12.138 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T09:15:12.150 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T09:15:12.360 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-13T09:15:12.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-09-13T09:15:13.877 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-09-13T09:15:13.879 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-09-13T09:15:42.393 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T09:15:42.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:15:42.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:15:43.882 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T09:15:43.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:15:43.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:16:00.957 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-13T09:16:02.237 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-09-13T09:16:09.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:16:09.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:16:11.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:16:11.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:16:13.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-13T09:16:14.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-09-13T09:16:25.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:16:25.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:16:26.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:16:26.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:16:28.097 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-13T09:16:43.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:16:43.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:16:44.737 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-09-13T09:16:46.173 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-13T09:16:59.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:16:59.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:17:00.245 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-09-13T09:17:00.828 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:17:00.828 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-13T09:17:00.829 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-09-13T09:17:00.835 INFO:teuthology.orchestra.run.smithi063.stdout:5.4.0-195-generic 2024-09-13T09:17:00.835 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T09:17:00.835 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:17:00.835 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:17:01.836 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T09:17:01.836 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:17:01.930 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-09-13T09:17:01.964 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:04.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:17:04.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:17:04.340 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-13T09:17:04.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:17:04.921 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-13T09:17:04.922 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-13T09:17:04.927 INFO:teuthology.orchestra.run.smithi099.stdout:5.4.0-195-generic 2024-09-13T09:17:04.928 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T09:17:04.928 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:17:04.928 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:17:05.928 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T09:17:05.929 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:17:06.024 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-09-13T09:17:06.052 DEBUG:teuthology.parallel:result is None 2024-09-13T09:17:06.053 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T09:17:06.061 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T09:17:06.061 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:17:06.064 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:17:06.071 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T09:17:06.104 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T09:17:06.147 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T09:17:06.147 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:17:06.150 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:17:06.178 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T09:17:06.188 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T09:17:06.188 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:17:06.190 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:17:06.206 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:17:06.208 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:17:06.217 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:17:06.218 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:17:06.220 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T09:17:06.229 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T09:17:06.230 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:17:06.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:17:06.278 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T09:17:06.287 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T09:17:06.287 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:17:06.290 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:17:06.327 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:17:06.334 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:17:06.384 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:17:06.384 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:17:06.403 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:17:06.410 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:17:06.417 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:17:06.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:17:06.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-09-13T09:17:06.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-13T09:17:06.532 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T09:17:06.539 INFO:teuthology.task.internal:Starting timer... 2024-09-13T09:17:06.539 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T09:17:06.624 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T09:17:06.665 DEBUG:teuthology.task.selinux:Excluding smithi063: OS 'ubuntu' does not support SELinux 2024-09-13T09:17:06.665 DEBUG:teuthology.task.selinux:Excluding smithi099: OS 'ubuntu' does not support SELinux 2024-09-13T09:17:06.665 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T09:17:06.665 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T09:17:06.665 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T09:17:06.666 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T09:17:06.707 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T09:17:06.710 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T09:17:06.850 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T09:17:06.861 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T09:17:06.862 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi099.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T09:22:43.921 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com')] 2024-09-13T09:22:43.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:22:43.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:22:44.015 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-09-13T09:22:44.138 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-09-13T09:22:44.139 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:22:44.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:22:44.223 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-13T09:22:44.346 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-13T09:22:44.346 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T09:22:44.357 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T09:22:44.357 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:22:44.357 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:22:44.360 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:22:44.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:22:44.394 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T09:22:44.395 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Command line: ntpd -gq 2024-09-13T09:22:44.395 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T09:22:44.395 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Command line: ntpd -gq 2024-09-13T09:22:44.395 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: proto: precision = 0.064 usec (-24) 2024-09-13T09:22:44.396 INFO:teuthology.orchestra.run.smithi099.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:22:44.396 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:22:44.396 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:22:44.397 INFO:teuthology.orchestra.run.smithi099.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:22:44.397 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listen normally on 3 enp3s0f1 172.21.15.99:123 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listen normally on 4 lo [::1]:123 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d803%5]:123 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:44 ntpd[13724]: Listening on routing socket on fd #22 for interface updates 2024-09-13T09:22:44.398 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: proto: precision = 0.123 usec (-23) 2024-09-13T09:22:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:22:44.399 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:22:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:22:44.400 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:22:44.400 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T09:22:44.400 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T09:22:44.400 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T09:22:44.400 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listen normally on 3 enp3s0f1 172.21.15.63:123 2024-09-13T09:22:44.401 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listen normally on 4 lo [::1]:123 2024-09-13T09:22:44.401 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8d7%5]:123 2024-09-13T09:22:44.401 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:44 ntpd[13671]: Listening on routing socket on fd #22 for interface updates 2024-09-13T09:22:45.395 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:45 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:45 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:45 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:45 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:45 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:45.396 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:45 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:47.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:47.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:47 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:48.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:48 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:48.398 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.398 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.398 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:48.399 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:48 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:49.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:49 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:49.616 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:49 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:49.616 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:49 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:49.616 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:49 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:49.616 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:49 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:49.616 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:49 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:49.619 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:49.619 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:49.619 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:49.619 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:49.619 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:49 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:50.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:50.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:50 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:50 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:50 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:50 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:50 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:50.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:50 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:50 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:51.395 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:51 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:51.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:51 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:51.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:51 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:51.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:51 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:51.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:51 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:51.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:51 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:51.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:51.397 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:51.398 INFO:teuthology.orchestra.run.smithi063.stdout:13 Sep 09:22:51 ntpd[13671]: ntpd: time slew +0.002099 s 2024-09-13T09:22:51.399 INFO:teuthology.orchestra.run.smithi063.stdout:ntpd: time slew +0.002099s 2024-09-13T09:22:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T09:22:51.399 INFO:teuthology.orchestra.run.smithi063.stderr:13 Sep 09:22:51 ntpd[13671]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T09:22:51.463 INFO:teuthology.orchestra.run.smithi063.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:22:51.463 INFO:teuthology.orchestra.run.smithi063.stdout:============================================================================== 2024-09-13T09:22:51.463 INFO:teuthology.orchestra.run.smithi063.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:51.464 INFO:teuthology.orchestra.run.smithi063.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:52.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:52.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:52.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:52.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:52.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:52.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:52 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:53.395 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:53 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:53.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:53 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:53.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:53 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:53.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:53 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:53.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:53 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:53.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:53 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:22:54.396 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:22:54.397 INFO:teuthology.orchestra.run.smithi099.stdout:13 Sep 09:22:54 ntpd[13724]: ntpd: time slew +0.000157 s 2024-09-13T09:22:54.397 INFO:teuthology.orchestra.run.smithi099.stdout:ntpd: time slew +0.000157s 2024-09-13T09:22:54.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T09:22:54.397 INFO:teuthology.orchestra.run.smithi099.stderr:13 Sep 09:22:54 ntpd[13724]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T09:22:54.473 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:22:54.474 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2024-09-13T09:22:54.474 INFO:teuthology.orchestra.run.smithi099.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:54.474 INFO:teuthology.orchestra.run.smithi099.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:54.474 INFO:teuthology.orchestra.run.smithi099.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:54.474 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:22:54.474 INFO:teuthology.run_tasks:Running task install... 2024-09-13T09:22:54.685 DEBUG:teuthology.task.install:project ceph 2024-09-13T09:22:54.685 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T09:22:54.685 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T09:22:54.685 INFO:teuthology.task.install:Using flavor: default 2024-09-13T09:22:54.691 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T09:22:54.691 INFO:teuthology.task.install:extra packages: [] 2024-09-13T09:22:54.691 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-key list | grep Ceph 2024-09-13T09:22:54.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-key list | grep Ceph 2024-09-13T09:22:54.790 INFO:teuthology.orchestra.run.smithi063.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T09:22:54.799 INFO:teuthology.orchestra.run.smithi099.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T09:22:54.853 INFO:teuthology.orchestra.run.smithi063.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T09:22:54.854 INFO:teuthology.orchestra.run.smithi063.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T09:22:54.855 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T09:22:54.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:22:54.860 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T09:22:54.860 INFO:teuthology.orchestra.run.smithi099.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T09:22:54.861 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T09:22:54.861 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:22:54.998 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default/ 2024-09-13T09:22:54.998 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1focal 2024-09-13T09:22:55.007 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default/ 2024-09-13T09:22:55.007 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1focal 2024-09-13T09:22:55.108 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:22:55.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T09:22:55.119 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:22:55.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T09:22:55.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-09-13T09:22:55.136 DEBUG:teuthology.orchestra.run.smithi099:> sudo apt-get update 2024-09-13T09:22:55.286 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T09:22:55.287 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T09:22:55.323 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal InRelease 2024-09-13T09:22:55.333 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal InRelease 2024-09-13T09:22:55.345 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T09:22:55.354 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T09:22:55.368 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T09:22:55.377 INFO:teuthology.orchestra.run.smithi099.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T09:22:55.384 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T09:22:55.389 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T09:22:55.393 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T09:22:55.398 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T09:22:55.426 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T09:22:55.432 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T09:22:55.467 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T09:22:55.474 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T09:22:55.549 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T09:22:55.556 INFO:teuthology.orchestra.run.smithi099.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T09:22:55.939 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 28.7 kB in 1s (44.7 kB/s) 2024-09-13T09:22:55.959 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 28.7 kB in 1s (42.7 kB/s) 2024-09-13T09:22:57.095 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-13T09:22:57.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1focal cephadm=18.2.4-820-g48a7de7e-1focal ceph-mds=18.2.4-820-g48a7de7e-1focal ceph-mgr=18.2.4-820-g48a7de7e-1focal ceph-common=18.2.4-820-g48a7de7e-1focal ceph-fuse=18.2.4-820-g48a7de7e-1focal ceph-test=18.2.4-820-g48a7de7e-1focal ceph-volume=18.2.4-820-g48a7de7e-1focal radosgw=18.2.4-820-g48a7de7e-1focal python3-rados=18.2.4-820-g48a7de7e-1focal python3-rgw=18.2.4-820-g48a7de7e-1focal python3-cephfs=18.2.4-820-g48a7de7e-1focal python3-rbd=18.2.4-820-g48a7de7e-1focal libcephfs2=18.2.4-820-g48a7de7e-1focal libcephfs-dev=18.2.4-820-g48a7de7e-1focal librados2=18.2.4-820-g48a7de7e-1focal librbd1=18.2.4-820-g48a7de7e-1focal rbd-fuse=18.2.4-820-g48a7de7e-1focal 2024-09-13T09:22:57.162 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-09-13T09:22:57.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1focal cephadm=18.2.4-820-g48a7de7e-1focal ceph-mds=18.2.4-820-g48a7de7e-1focal ceph-mgr=18.2.4-820-g48a7de7e-1focal ceph-common=18.2.4-820-g48a7de7e-1focal ceph-fuse=18.2.4-820-g48a7de7e-1focal ceph-test=18.2.4-820-g48a7de7e-1focal ceph-volume=18.2.4-820-g48a7de7e-1focal radosgw=18.2.4-820-g48a7de7e-1focal python3-rados=18.2.4-820-g48a7de7e-1focal python3-rgw=18.2.4-820-g48a7de7e-1focal python3-cephfs=18.2.4-820-g48a7de7e-1focal python3-rbd=18.2.4-820-g48a7de7e-1focal libcephfs2=18.2.4-820-g48a7de7e-1focal libcephfs-dev=18.2.4-820-g48a7de7e-1focal librados2=18.2.4-820-g48a7de7e-1focal librbd1=18.2.4-820-g48a7de7e-1focal rbd-fuse=18.2.4-820-g48a7de7e-1focal 2024-09-13T09:22:57.195 INFO:teuthology.orchestra.run.smithi099.stdout:Reading package lists... 2024-09-13T09:22:57.262 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-09-13T09:22:57.413 INFO:teuthology.orchestra.run.smithi099.stdout:Building dependency tree... 2024-09-13T09:22:57.414 INFO:teuthology.orchestra.run.smithi099.stdout:Reading state information... 2024-09-13T09:22:57.487 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-09-13T09:22:57.488 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-09-13T09:22:57.609 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:22:57.610 INFO:teuthology.orchestra.run.smithi099.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T09:22:57.610 INFO:teuthology.orchestra.run.smithi099.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:22:57.610 INFO:teuthology.orchestra.run.smithi099.stdout:The following additional packages will be installed: 2024-09-13T09:22:57.610 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T09:22:57.610 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T09:22:57.610 INFO:teuthology.orchestra.run.smithi099.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T09:22:57.611 INFO:teuthology.orchestra.run.smithi099.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T09:22:57.611 INFO:teuthology.orchestra.run.smithi099.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T09:22:57.611 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T09:22:57.612 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout:Suggested packages: 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T09:22:57.613 INFO:teuthology.orchestra.run.smithi099.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T09:22:57.614 INFO:teuthology.orchestra.run.smithi099.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T09:22:57.614 INFO:teuthology.orchestra.run.smithi099.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T09:22:57.614 INFO:teuthology.orchestra.run.smithi099.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T09:22:57.614 INFO:teuthology.orchestra.run.smithi099.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T09:22:57.614 INFO:teuthology.orchestra.run.smithi099.stdout:Recommended packages: 2024-09-13T09:22:57.614 INFO:teuthology.orchestra.run.smithi099.stdout: btrfs-tools 2024-09-13T09:22:57.701 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:22:57.702 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T09:22:57.702 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:22:57.702 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-09-13T09:22:57.703 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T09:22:57.703 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T09:22:57.703 INFO:teuthology.orchestra.run.smithi063.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T09:22:57.703 INFO:teuthology.orchestra.run.smithi063.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T09:22:57.704 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T09:22:57.705 INFO:teuthology.orchestra.run.smithi063.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout:Recommended packages: 2024-09-13T09:22:57.706 INFO:teuthology.orchestra.run.smithi063.stdout: btrfs-tools 2024-09-13T09:22:57.730 INFO:teuthology.orchestra.run.smithi099.stdout:The following NEW packages will be installed: 2024-09-13T09:22:57.730 INFO:teuthology.orchestra.run.smithi099.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T09:22:57.730 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T09:22:57.730 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T09:22:57.731 INFO:teuthology.orchestra.run.smithi099.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-13T09:22:57.731 INFO:teuthology.orchestra.run.smithi099.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-13T09:22:57.731 INFO:teuthology.orchestra.run.smithi099.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T09:22:57.731 INFO:teuthology.orchestra.run.smithi099.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-13T09:22:57.732 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout:The following packages will be upgraded: 2024-09-13T09:22:57.733 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 librbd1 2024-09-13T09:22:57.826 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-09-13T09:22:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T09:22:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T09:22:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T09:22:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-13T09:22:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-13T09:22:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T09:22:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-13T09:22:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-13T09:22:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T09:22:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-13T09:22:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-13T09:22:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-13T09:22:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet 2024-09-13T09:22:57.829 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages will be upgraded: 2024-09-13T09:22:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 librbd1 2024-09-13T09:22:57.883 INFO:teuthology.orchestra.run.smithi099.stdout:2 upgraded, 110 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T09:22:57.883 INFO:teuthology.orchestra.run.smithi099.stdout:Need to get 199 MB of archives. 2024-09-13T09:22:57.883 INFO:teuthology.orchestra.run.smithi099.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-13T09:22:57.883 INFO:teuthology.orchestra.run.smithi099.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1focal [3,456 kB] 2024-09-13T09:22:57.927 INFO:teuthology.orchestra.run.smithi099.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T09:22:57.979 INFO:teuthology.orchestra.run.smithi063.stdout:2 upgraded, 110 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T09:22:57.979 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 199 MB of archives. 2024-09-13T09:22:57.979 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-13T09:22:57.979 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1focal [3,456 kB] 2024-09-13T09:22:58.023 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T09:22:58.075 INFO:teuthology.orchestra.run.smithi099.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1focal [4,039 kB] 2024-09-13T09:22:58.180 INFO:teuthology.orchestra.run.smithi099.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1focal [746 kB] 2024-09-13T09:22:58.191 INFO:teuthology.orchestra.run.smithi099.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1focal [387 kB] 2024-09-13T09:22:58.200 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1focal [4,039 kB] 2024-09-13T09:22:58.205 INFO:teuthology.orchestra.run.smithi099.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1focal [31.9 kB] 2024-09-13T09:22:58.206 INFO:teuthology.orchestra.run.smithi099.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1focal [200 kB] 2024-09-13T09:22:58.217 INFO:teuthology.orchestra.run.smithi099.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1focal [62.5 kB] 2024-09-13T09:22:58.218 INFO:teuthology.orchestra.run.smithi099.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1focal [367 kB] 2024-09-13T09:22:58.224 INFO:teuthology.orchestra.run.smithi099.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1focal [6,628 kB] 2024-09-13T09:22:58.328 INFO:teuthology.orchestra.run.smithi099.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T09:22:58.342 INFO:teuthology.orchestra.run.smithi099.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T09:22:58.346 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1focal [746 kB] 2024-09-13T09:22:58.359 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1focal [387 kB] 2024-09-13T09:22:58.388 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1focal [31.9 kB] 2024-09-13T09:22:58.389 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1focal [200 kB] 2024-09-13T09:22:58.393 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1focal [62.5 kB] 2024-09-13T09:22:58.394 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1focal [367 kB] 2024-09-13T09:22:58.399 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1focal [6,628 kB] 2024-09-13T09:22:58.420 INFO:teuthology.orchestra.run.smithi099.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1focal [113 kB] 2024-09-13T09:22:58.421 INFO:teuthology.orchestra.run.smithi099.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1focal [478 kB] 2024-09-13T09:22:58.422 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T09:22:58.426 INFO:teuthology.orchestra.run.smithi099.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T09:22:58.435 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T09:22:58.437 INFO:teuthology.orchestra.run.smithi099.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1focal [25.2 MB] 2024-09-13T09:22:58.519 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T09:22:58.643 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1focal [113 kB] 2024-09-13T09:22:58.644 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1focal [478 kB] 2024-09-13T09:22:58.668 INFO:teuthology.orchestra.run.smithi099.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T09:22:58.677 INFO:teuthology.orchestra.run.smithi099.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T09:22:58.683 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1focal [25.2 MB] 2024-09-13T09:22:58.705 INFO:teuthology.orchestra.run.smithi099.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T09:22:58.713 INFO:teuthology.orchestra.run.smithi099.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T09:22:58.714 INFO:teuthology.orchestra.run.smithi099.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T09:22:58.718 INFO:teuthology.orchestra.run.smithi099.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T09:22:58.719 INFO:teuthology.orchestra.run.smithi099.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T09:22:58.732 INFO:teuthology.orchestra.run.smithi099.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T09:22:58.733 INFO:teuthology.orchestra.run.smithi099.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T09:22:58.750 INFO:teuthology.orchestra.run.smithi099.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T09:22:58.763 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T09:22:58.773 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T09:22:58.800 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T09:22:58.809 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T09:22:58.810 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T09:22:58.813 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T09:22:58.814 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T09:22:58.820 INFO:teuthology.orchestra.run.smithi099.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T09:22:58.821 INFO:teuthology.orchestra.run.smithi099.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T09:22:58.822 INFO:teuthology.orchestra.run.smithi099.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T09:22:58.824 INFO:teuthology.orchestra.run.smithi099.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T09:22:58.824 INFO:teuthology.orchestra.run.smithi099.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T09:22:58.825 INFO:teuthology.orchestra.run.smithi099.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T09:22:58.826 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T09:22:58.828 INFO:teuthology.orchestra.run.smithi099.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T09:22:58.829 INFO:teuthology.orchestra.run.smithi099.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T09:22:58.829 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T09:22:58.832 INFO:teuthology.orchestra.run.smithi099.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T09:22:58.846 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T09:22:58.913 INFO:teuthology.orchestra.run.smithi099.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T09:22:58.913 INFO:teuthology.orchestra.run.smithi099.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T09:22:58.916 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T09:22:58.916 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T09:22:58.917 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T09:22:58.918 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T09:22:58.919 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T09:22:58.919 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T09:22:58.921 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T09:22:58.922 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T09:22:58.924 INFO:teuthology.orchestra.run.smithi099.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T09:22:58.925 INFO:teuthology.orchestra.run.smithi099.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T09:22:58.925 INFO:teuthology.orchestra.run.smithi099.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T09:22:58.926 INFO:teuthology.orchestra.run.smithi099.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T09:22:58.927 INFO:teuthology.orchestra.run.smithi099.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T09:22:58.928 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T09:22:58.929 INFO:teuthology.orchestra.run.smithi099.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T09:22:58.994 INFO:teuthology.orchestra.run.smithi099.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T09:22:59.000 INFO:teuthology.orchestra.run.smithi099.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T09:22:59.008 INFO:teuthology.orchestra.run.smithi063.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T09:22:59.009 INFO:teuthology.orchestra.run.smithi063.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T09:22:59.016 INFO:teuthology.orchestra.run.smithi063.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T09:22:59.016 INFO:teuthology.orchestra.run.smithi063.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T09:22:59.017 INFO:teuthology.orchestra.run.smithi063.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T09:22:59.017 INFO:teuthology.orchestra.run.smithi063.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T09:22:59.019 INFO:teuthology.orchestra.run.smithi063.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T09:22:59.022 INFO:teuthology.orchestra.run.smithi063.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T09:22:59.075 INFO:teuthology.orchestra.run.smithi099.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T09:22:59.079 INFO:teuthology.orchestra.run.smithi099.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T09:22:59.081 INFO:teuthology.orchestra.run.smithi099.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T09:22:59.082 INFO:teuthology.orchestra.run.smithi099.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T09:22:59.083 INFO:teuthology.orchestra.run.smithi099.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T09:22:59.083 INFO:teuthology.orchestra.run.smithi099.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T09:22:59.085 INFO:teuthology.orchestra.run.smithi099.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T09:22:59.086 INFO:teuthology.orchestra.run.smithi099.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T09:22:59.089 INFO:teuthology.orchestra.run.smithi063.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T09:22:59.093 INFO:teuthology.orchestra.run.smithi063.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T09:22:59.157 INFO:teuthology.orchestra.run.smithi099.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T09:22:59.157 INFO:teuthology.orchestra.run.smithi099.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T09:22:59.169 INFO:teuthology.orchestra.run.smithi099.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T09:22:59.169 INFO:teuthology.orchestra.run.smithi099.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T09:22:59.171 INFO:teuthology.orchestra.run.smithi063.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T09:22:59.171 INFO:teuthology.orchestra.run.smithi099.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T09:22:59.173 INFO:teuthology.orchestra.run.smithi099.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T09:22:59.173 INFO:teuthology.orchestra.run.smithi063.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T09:22:59.174 INFO:teuthology.orchestra.run.smithi063.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T09:22:59.175 INFO:teuthology.orchestra.run.smithi063.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T09:22:59.176 INFO:teuthology.orchestra.run.smithi063.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T09:22:59.176 INFO:teuthology.orchestra.run.smithi063.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T09:22:59.178 INFO:teuthology.orchestra.run.smithi063.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T09:22:59.178 INFO:teuthology.orchestra.run.smithi063.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T09:22:59.239 INFO:teuthology.orchestra.run.smithi099.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T09:22:59.240 INFO:teuthology.orchestra.run.smithi099.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T09:22:59.241 INFO:teuthology.orchestra.run.smithi099.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T09:22:59.252 INFO:teuthology.orchestra.run.smithi063.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T09:22:59.252 INFO:teuthology.orchestra.run.smithi063.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T09:22:59.260 INFO:teuthology.orchestra.run.smithi063.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T09:22:59.261 INFO:teuthology.orchestra.run.smithi063.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T09:22:59.262 INFO:teuthology.orchestra.run.smithi063.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T09:22:59.264 INFO:teuthology.orchestra.run.smithi063.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T09:22:59.334 INFO:teuthology.orchestra.run.smithi063.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T09:22:59.335 INFO:teuthology.orchestra.run.smithi063.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T09:22:59.336 INFO:teuthology.orchestra.run.smithi063.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T09:22:59.355 INFO:teuthology.orchestra.run.smithi099.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1focal [9,612 kB] 2024-09-13T09:22:59.521 INFO:teuthology.orchestra.run.smithi099.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T09:22:59.572 INFO:teuthology.orchestra.run.smithi063.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1focal [9,612 kB] 2024-09-13T09:22:59.619 INFO:teuthology.orchestra.run.smithi063.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T09:22:59.631 INFO:teuthology.orchestra.run.smithi099.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T09:22:59.631 INFO:teuthology.orchestra.run.smithi099.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T09:22:59.635 INFO:teuthology.orchestra.run.smithi099.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1focal [148 kB] 2024-09-13T09:22:59.635 INFO:teuthology.orchestra.run.smithi099.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T09:22:59.637 INFO:teuthology.orchestra.run.smithi099.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1focal [1,575 kB] 2024-09-13T09:22:59.654 INFO:teuthology.orchestra.run.smithi099.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1focal [6,640 kB] 2024-09-13T09:22:59.669 INFO:teuthology.orchestra.run.smithi099.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T09:22:59.669 INFO:teuthology.orchestra.run.smithi099.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T09:22:59.669 INFO:teuthology.orchestra.run.smithi099.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T09:22:59.670 INFO:teuthology.orchestra.run.smithi099.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T09:22:59.670 INFO:teuthology.orchestra.run.smithi099.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T09:22:59.671 INFO:teuthology.orchestra.run.smithi099.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T09:22:59.671 INFO:teuthology.orchestra.run.smithi099.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T09:22:59.693 INFO:teuthology.orchestra.run.smithi063.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T09:22:59.695 INFO:teuthology.orchestra.run.smithi063.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T09:22:59.713 INFO:teuthology.orchestra.run.smithi099.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T09:22:59.754 INFO:teuthology.orchestra.run.smithi099.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T09:22:59.755 INFO:teuthology.orchestra.run.smithi099.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T09:22:59.761 INFO:teuthology.orchestra.run.smithi099.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T09:22:59.763 INFO:teuthology.orchestra.run.smithi099.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T09:22:59.764 INFO:teuthology.orchestra.run.smithi099.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T09:22:59.769 INFO:teuthology.orchestra.run.smithi099.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T09:22:59.770 INFO:teuthology.orchestra.run.smithi063.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T09:22:59.799 INFO:teuthology.orchestra.run.smithi099.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T09:22:59.805 INFO:teuthology.orchestra.run.smithi099.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T09:22:59.819 INFO:teuthology.orchestra.run.smithi099.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1focal [24.5 MB] 2024-09-13T09:22:59.823 INFO:teuthology.orchestra.run.smithi063.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T09:22:59.824 INFO:teuthology.orchestra.run.smithi063.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T09:22:59.843 INFO:teuthology.orchestra.run.smithi099.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T09:22:59.844 INFO:teuthology.orchestra.run.smithi063.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T09:22:59.848 INFO:teuthology.orchestra.run.smithi063.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1focal [148 kB] 2024-09-13T09:22:59.851 INFO:teuthology.orchestra.run.smithi063.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1focal [1,575 kB] 2024-09-13T09:22:59.881 INFO:teuthology.orchestra.run.smithi099.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T09:22:59.885 INFO:teuthology.orchestra.run.smithi099.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T09:22:59.901 INFO:teuthology.orchestra.run.smithi063.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1focal [6,640 kB] 2024-09-13T09:22:59.904 INFO:teuthology.orchestra.run.smithi063.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T09:22:59.909 INFO:teuthology.orchestra.run.smithi063.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T09:22:59.913 INFO:teuthology.orchestra.run.smithi063.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T09:22:59.916 INFO:teuthology.orchestra.run.smithi063.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T09:22:59.918 INFO:teuthology.orchestra.run.smithi063.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T09:22:59.922 INFO:teuthology.orchestra.run.smithi063.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T09:22:59.924 INFO:teuthology.orchestra.run.smithi063.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T09:22:59.997 INFO:teuthology.orchestra.run.smithi063.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T09:23:00.002 INFO:teuthology.orchestra.run.smithi063.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T09:23:00.004 INFO:teuthology.orchestra.run.smithi063.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T09:23:00.048 INFO:teuthology.orchestra.run.smithi099.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T09:23:00.050 INFO:teuthology.orchestra.run.smithi099.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T09:23:00.051 INFO:teuthology.orchestra.run.smithi099.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T09:23:00.052 INFO:teuthology.orchestra.run.smithi099.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T09:23:00.064 INFO:teuthology.orchestra.run.smithi099.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T09:23:00.065 INFO:teuthology.orchestra.run.smithi099.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T09:23:00.082 INFO:teuthology.orchestra.run.smithi099.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T09:23:00.083 INFO:teuthology.orchestra.run.smithi099.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T09:23:00.083 INFO:teuthology.orchestra.run.smithi099.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T09:23:00.155 INFO:teuthology.orchestra.run.smithi063.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1focal [24.5 MB] 2024-09-13T09:23:00.173 INFO:teuthology.orchestra.run.smithi063.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T09:23:00.174 INFO:teuthology.orchestra.run.smithi063.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T09:23:00.174 INFO:teuthology.orchestra.run.smithi063.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T09:23:00.222 INFO:teuthology.orchestra.run.smithi063.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T09:23:00.243 INFO:teuthology.orchestra.run.smithi063.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T09:23:00.243 INFO:teuthology.orchestra.run.smithi063.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T09:23:00.244 INFO:teuthology.orchestra.run.smithi063.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T09:23:00.245 INFO:teuthology.orchestra.run.smithi063.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T09:23:00.245 INFO:teuthology.orchestra.run.smithi063.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T09:23:00.654 INFO:teuthology.orchestra.run.smithi099.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1focal [13.8 kB] 2024-09-13T09:23:00.655 INFO:teuthology.orchestra.run.smithi099.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1focal [855 kB] 2024-09-13T09:23:00.668 INFO:teuthology.orchestra.run.smithi099.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1focal [2,295 kB] 2024-09-13T09:23:00.684 INFO:teuthology.orchestra.run.smithi099.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T09:23:00.686 INFO:teuthology.orchestra.run.smithi099.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1focal [143 kB] 2024-09-13T09:23:00.690 INFO:teuthology.orchestra.run.smithi099.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1focal [7,015 kB] 2024-09-13T09:23:00.902 INFO:teuthology.orchestra.run.smithi099.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1focal [7,868 kB] 2024-09-13T09:23:01.097 INFO:teuthology.orchestra.run.smithi063.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1focal [13.8 kB] 2024-09-13T09:23:01.097 INFO:teuthology.orchestra.run.smithi063.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1focal [855 kB] 2024-09-13T09:23:01.109 INFO:teuthology.orchestra.run.smithi099.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1focal [13.9 kB] 2024-09-13T09:23:01.109 INFO:teuthology.orchestra.run.smithi099.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1focal [46.9 MB] 2024-09-13T09:23:01.110 INFO:teuthology.orchestra.run.smithi063.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1focal [2,295 kB] 2024-09-13T09:23:01.141 INFO:teuthology.orchestra.run.smithi063.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1focal [238 kB] 2024-09-13T09:23:01.145 INFO:teuthology.orchestra.run.smithi063.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1focal [143 kB] 2024-09-13T09:23:01.147 INFO:teuthology.orchestra.run.smithi063.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1focal [7,015 kB] 2024-09-13T09:23:01.404 INFO:teuthology.orchestra.run.smithi063.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1focal [7,868 kB] 2024-09-13T09:23:01.691 INFO:teuthology.orchestra.run.smithi063.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1focal [13.9 kB] 2024-09-13T09:23:01.691 INFO:teuthology.orchestra.run.smithi063.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1focal [46.9 MB] 2024-09-13T09:23:02.757 INFO:teuthology.orchestra.run.smithi099.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1focal [120 kB] 2024-09-13T09:23:02.760 INFO:teuthology.orchestra.run.smithi099.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1focal [37.4 kB] 2024-09-13T09:23:02.761 INFO:teuthology.orchestra.run.smithi099.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1focal [13.5 MB] 2024-09-13T09:23:03.164 INFO:teuthology.orchestra.run.smithi099.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1focal [107 kB] 2024-09-13T09:23:03.353 INFO:teuthology.orchestra.run.smithi063.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1focal [120 kB] 2024-09-13T09:23:03.355 INFO:teuthology.orchestra.run.smithi063.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1focal [37.4 kB] 2024-09-13T09:23:03.356 INFO:teuthology.orchestra.run.smithi063.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1focal [13.5 MB] 2024-09-13T09:23:03.532 INFO:teuthology.orchestra.run.smithi063.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1focal [107 kB] 2024-09-13T09:23:03.806 INFO:teuthology.orchestra.run.smithi099.stdout:Fetched 199 MB in 5s (36.7 MB/s) 2024-09-13T09:23:04.025 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T09:23:04.083 INFO:teuthology.orchestra.run.smithi099.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T09:23:04.088 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T09:23:04.174 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T09:23:04.187 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 199 MB in 6s (34.9 MB/s) 2024-09-13T09:23:04.359 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T09:23:04.416 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T09:23:04.420 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T09:23:04.500 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T09:23:04.510 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T09:23:04.530 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T09:23:04.560 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T09:23:04.852 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T09:23:04.870 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T09:23:04.896 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T09:23:04.910 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T09:23:04.915 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T09:23:04.971 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T09:23:05.272 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T09:23:05.288 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T09:23:05.330 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T09:23:05.450 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T09:23:05.468 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T09:23:05.552 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:23:05.784 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T09:23:05.799 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T09:23:05.852 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:23:06.086 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T09:23:06.101 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T09:23:06.169 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:23:06.353 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T09:23:06.369 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T09:23:06.411 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:23:06.556 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T09:23:06.573 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T09:23:06.614 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:23:06.739 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T09:23:06.757 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T09:23:06.789 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:23:07.042 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T09:23:07.058 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T09:23:07.117 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T09:23:07.142 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T09:23:07.161 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T09:23:07.200 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T09:23:07.505 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../007-librbd1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:07.620 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T09:23:07.686 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../007-librbd1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:07.812 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T09:23:08.343 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:08.450 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:08.466 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T09:23:08.549 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T09:23:09.194 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T09:23:09.210 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:09.245 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T09:23:09.261 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:09.269 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:09.302 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:09.630 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rados. 2024-09-13T09:23:09.650 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../010-python3-rados_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:09.688 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:09.763 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rados. 2024-09-13T09:23:09.779 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../010-python3-rados_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:09.821 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:10.083 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T09:23:10.100 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:10.132 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T09:23:10.150 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:10.166 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:10.190 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:10.401 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T09:23:10.410 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T09:23:10.417 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:10.425 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:10.451 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:10.485 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:10.746 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T09:23:10.764 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:10.787 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T09:23:10.806 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:10.829 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:10.845 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:11.173 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T09:23:11.190 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T09:23:11.191 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T09:23:11.206 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T09:23:11.231 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T09:23:11.249 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T09:23:11.467 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T09:23:11.485 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:11.509 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T09:23:11.525 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:11.527 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:11.567 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:11.870 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T09:23:11.887 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T09:23:11.928 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T09:23:11.937 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T09:23:11.953 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T09:23:12.012 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T09:23:12.264 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T09:23:12.283 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T09:23:12.314 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T09:23:12.390 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T09:23:12.410 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T09:23:12.448 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T09:23:12.769 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T09:23:12.787 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T09:23:12.875 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T09:23:12.984 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T09:23:13.000 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T09:23:13.042 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T09:23:13.220 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librgw2. 2024-09-13T09:23:13.236 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:13.278 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:13.395 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package librgw2. 2024-09-13T09:23:13.414 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:13.453 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:14.147 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T09:23:14.166 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:14.205 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:14.347 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T09:23:14.367 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:14.437 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:14.558 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liboath0. 2024-09-13T09:23:14.577 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T09:23:14.616 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T09:23:14.825 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liboath0. 2024-09-13T09:23:14.841 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T09:23:14.883 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T09:23:14.902 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T09:23:14.922 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:14.960 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:15.203 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T09:23:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:15.413 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-common. 2024-09-13T09:23:15.429 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:15.471 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:15.772 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-common. 2024-09-13T09:23:15.788 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:15.830 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:17.927 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-base. 2024-09-13T09:23:17.946 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:17.996 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:18.277 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-base. 2024-09-13T09:23:18.293 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:18.346 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:19.248 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T09:23:19.267 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T09:23:19.306 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T09:23:19.634 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T09:23:19.640 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T09:23:19.651 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T09:23:19.659 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T09:23:19.692 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T09:23:19.698 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T09:23:20.003 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T09:23:20.020 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T09:23:20.061 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T09:23:20.085 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T09:23:20.100 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T09:23:20.143 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T09:23:20.348 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T09:23:20.366 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T09:23:20.414 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T09:23:20.454 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T09:23:20.472 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T09:23:20.504 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T09:23:20.717 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-mako. 2024-09-13T09:23:20.733 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T09:23:20.775 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T09:23:20.840 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T09:23:20.856 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T09:23:20.915 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T09:23:21.111 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T09:23:21.128 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T09:23:21.169 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:23:21.243 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-mako. 2024-09-13T09:23:21.260 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T09:23:21.301 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T09:23:21.472 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T09:23:21.491 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T09:23:21.530 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T09:23:21.646 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T09:23:21.663 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T09:23:21.704 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:23:21.833 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webob. 2024-09-13T09:23:21.849 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T09:23:21.883 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T09:23:22.049 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T09:23:22.065 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T09:23:22.123 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T09:23:22.253 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T09:23:22.271 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T09:23:22.303 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T09:23:22.451 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webob. 2024-09-13T09:23:22.467 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T09:23:22.509 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T09:23:22.614 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T09:23:22.631 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T09:23:22.672 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T09:23:22.862 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T09:23:22.882 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T09:23:22.920 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T09:23:23.025 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T09:23:23.041 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T09:23:23.095 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T09:23:23.419 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T09:23:23.436 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T09:23:23.477 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T09:23:23.623 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T09:23:23.640 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T09:23:23.706 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T09:23:23.780 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-paste. 2024-09-13T09:23:23.796 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T09:23:23.830 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T09:23:24.084 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T09:23:24.100 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T09:23:24.145 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T09:23:24.158 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T09:23:24.175 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T09:23:24.216 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T09:23:24.495 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T09:23:24.502 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T09:23:24.512 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T09:23:24.522 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T09:23:24.553 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T09:23:24.560 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T09:23:24.880 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T09:23:24.898 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T09:23:24.915 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-paste. 2024-09-13T09:23:24.935 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T09:23:24.940 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T09:23:24.973 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T09:23:25.249 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T09:23:25.267 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T09:23:25.307 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T09:23:25.308 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T09:23:25.324 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T09:23:25.366 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T09:23:25.650 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T09:23:25.670 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T09:23:25.694 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T09:23:25.708 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T09:23:25.714 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T09:23:25.752 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T09:23:26.003 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T09:23:26.019 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:26.061 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:26.063 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T09:23:26.083 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T09:23:26.121 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T09:23:26.432 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T09:23:26.449 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:26.490 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:26.491 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T09:23:26.510 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T09:23:26.549 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T09:23:26.810 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T09:23:26.830 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T09:23:26.868 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T09:23:26.887 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T09:23:26.903 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T09:23:26.945 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T09:23:27.154 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T09:23:27.174 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T09:23:27.212 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T09:23:27.265 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T09:23:27.281 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:27.323 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:27.566 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T09:23:27.586 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T09:23:27.632 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T09:23:27.746 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T09:23:27.762 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:27.805 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:27.994 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T09:23:28.011 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:28.052 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:28.185 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T09:23:28.203 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T09:23:28.234 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T09:23:29.277 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T09:23:29.295 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:29.335 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:29.438 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T09:23:29.453 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T09:23:29.495 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T09:23:29.926 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T09:23:29.946 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T09:23:29.989 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T09:23:30.239 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T09:23:30.258 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:30.297 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:30.334 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T09:23:30.352 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:30.392 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:30.945 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T09:23:30.961 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:31.003 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:31.982 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T09:23:31.998 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:32.040 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:32.474 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph. 2024-09-13T09:23:32.490 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../051-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:32.531 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:32.859 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T09:23:32.878 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:32.917 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:33.362 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T09:23:33.380 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:33.420 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:33.906 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package cephadm. 2024-09-13T09:23:33.926 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../054-cephadm_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:33.956 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:34.336 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph. 2024-09-13T09:23:34.352 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../051-ceph_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:34.368 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T09:23:34.384 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T09:23:34.418 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:34.435 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T09:23:34.771 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T09:23:34.788 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:34.829 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:34.872 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T09:23:34.890 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T09:23:34.929 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T09:23:35.233 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T09:23:35.251 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:35.282 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T09:23:35.282 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:35.298 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:35.365 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:35.586 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T09:23:35.606 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T09:23:35.635 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T09:23:35.910 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package cephadm. 2024-09-13T09:23:35.925 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../054-cephadm_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:35.930 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-attr. 2024-09-13T09:23:35.947 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-13T09:23:35.988 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T09:23:36.009 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:36.324 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T09:23:36.343 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T09:23:36.345 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T09:23:36.361 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T09:23:36.383 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T09:23:36.403 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T09:23:36.705 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-py. 2024-09-13T09:23:36.720 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T09:23:36.760 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T09:23:36.807 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T09:23:36.826 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T09:23:36.864 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T09:23:37.097 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T09:23:37.113 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T09:23:37.155 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T09:23:37.218 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T09:23:37.236 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:37.292 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:37.441 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T09:23:37.457 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-13T09:23:37.499 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T09:23:37.654 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T09:23:37.670 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T09:23:37.712 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T09:23:37.801 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T09:23:37.821 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T09:23:37.859 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T09:23:38.023 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-attr. 2024-09-13T09:23:38.039 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-13T09:23:38.081 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T09:23:38.162 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T09:23:38.179 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T09:23:38.220 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T09:23:38.473 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T09:23:38.491 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T09:23:38.534 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T09:23:38.558 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T09:23:38.576 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T09:23:38.616 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T09:23:38.853 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T09:23:38.869 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T09:23:38.870 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-py. 2024-09-13T09:23:38.888 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T09:23:38.929 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T09:23:38.935 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T09:23:39.230 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T09:23:39.250 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T09:23:39.274 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T09:23:39.287 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T09:23:39.294 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T09:23:39.324 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T09:23:39.598 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-routes. 2024-09-13T09:23:39.618 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-13T09:23:39.635 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T09:23:39.652 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-13T09:23:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T09:23:39.693 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T09:23:39.984 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T09:23:40.001 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:40.013 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T09:23:40.029 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T09:23:40.042 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:40.079 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T09:23:40.415 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T09:23:40.432 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T09:23:40.473 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T09:23:40.827 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T09:23:40.844 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T09:23:40.885 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T09:23:41.137 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T09:23:41.153 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T09:23:41.195 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T09:23:41.339 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T09:23:41.356 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T09:23:41.397 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T09:23:41.473 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T09:23:41.490 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T09:23:41.531 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T09:23:41.700 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T09:23:41.717 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T09:23:41.783 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T09:23:41.925 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-routes. 2024-09-13T09:23:41.944 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-13T09:23:41.992 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T09:23:42.119 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T09:23:42.136 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T09:23:42.185 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T09:23:42.311 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T09:23:42.331 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:42.369 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:43.485 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T09:23:43.503 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T09:23:43.544 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T09:23:43.867 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T09:23:43.882 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T09:23:43.924 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T09:23:44.122 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T09:23:44.142 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T09:23:44.180 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T09:23:44.202 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T09:23:44.221 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T09:23:44.260 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T09:23:44.604 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T09:23:44.621 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T09:23:44.662 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T09:23:44.959 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T09:23:44.976 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T09:23:45.016 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T09:23:45.701 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T09:23:45.717 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:45.758 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:45.997 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T09:23:46.016 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T09:23:46.055 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T09:23:46.634 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T09:23:46.652 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T09:23:46.691 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T09:23:46.862 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T09:23:46.880 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T09:23:46.919 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T09:23:47.070 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T09:23:47.088 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T09:23:47.161 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T09:23:47.222 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T09:23:47.240 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T09:23:47.280 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T09:23:47.626 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T09:23:47.646 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T09:23:47.684 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T09:23:47.920 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T09:23:47.936 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:47.978 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:48.072 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T09:23:48.089 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-13T09:23:48.130 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T09:23:48.454 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T09:23:48.474 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T09:23:48.512 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T09:23:48.833 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T09:23:48.849 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T09:23:48.891 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T09:23:49.158 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T09:23:49.176 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T09:23:49.194 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T09:23:49.212 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T09:23:49.217 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T09:23:49.252 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T09:23:49.554 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T09:23:49.569 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T09:23:49.589 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T09:23:49.605 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T09:23:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T09:23:49.647 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T09:23:49.959 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T09:23:49.973 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T09:23:49.976 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T09:23:49.989 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T09:23:50.017 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T09:23:50.056 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T09:23:50.353 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T09:23:50.372 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T09:23:50.411 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T09:23:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T09:23:50.504 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-13T09:23:50.544 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T09:23:50.874 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T09:23:50.890 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T09:23:50.944 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T09:23:50.956 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T09:23:50.961 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:51.002 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:51.302 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T09:23:51.323 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T09:23:51.339 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T09:23:51.356 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T09:23:51.360 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T09:23:51.397 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T09:23:51.692 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T09:23:51.709 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T09:23:51.749 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T09:23:51.750 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T09:23:51.766 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T09:23:51.807 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:23:52.079 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package jq. 2024-09-13T09:23:52.096 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T09:23:52.137 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:23:52.162 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T09:23:52.180 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T09:23:52.220 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T09:23:52.390 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package socat. 2024-09-13T09:23:52.406 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T09:23:52.448 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T09:23:52.564 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T09:23:52.581 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T09:23:52.622 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T09:23:52.759 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T09:23:52.780 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T09:23:52.817 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T09:23:52.969 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T09:23:52.986 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T09:23:53.052 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T09:23:53.188 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-test. 2024-09-13T09:23:53.205 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../094-ceph-test_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:53.246 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:53.657 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T09:23:53.676 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:53.715 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:54.076 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T09:23:54.093 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T09:23:54.134 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T09:23:54.512 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T09:23:54.532 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T09:23:54.604 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:23:55.123 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package jq. 2024-09-13T09:23:55.143 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T09:23:55.181 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:23:55.476 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package socat. 2024-09-13T09:23:55.492 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T09:23:55.559 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T09:23:55.912 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T09:23:55.929 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T09:23:55.970 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T09:23:56.341 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-test. 2024-09-13T09:23:56.360 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../094-ceph-test_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:56.398 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:57.350 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T09:23:57.370 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:23:57.408 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:57.703 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T09:23:57.721 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T09:23:57.770 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:23:58.249 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T09:23:58.267 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:23:58.308 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:23:58.709 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T09:23:58.729 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T09:23:58.777 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T09:23:59.100 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T09:23:59.117 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T09:23:59.158 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T09:23:59.545 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T09:23:59.563 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T09:23:59.602 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:00.513 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T09:24:00.530 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-820-g48a7de7e-1focal_all.deb ... 2024-09-13T09:24:00.571 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:00.884 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T09:24:00.901 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T09:24:00.942 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:01.236 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T09:24:01.254 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T09:24:01.293 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T09:24:01.529 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T09:24:01.546 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:24:01.587 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:01.630 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T09:24:01.647 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T09:24:01.688 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T09:24:01.974 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T09:24:01.995 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T09:24:02.029 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T09:24:02.041 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T09:24:02.045 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T09:24:02.105 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T09:24:02.394 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T09:24:02.411 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T09:24:02.416 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T09:24:02.435 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T09:24:02.452 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T09:24:02.466 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T09:24:02.847 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T09:24:02.865 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T09:24:02.914 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T09:24:02.930 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:02.934 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T09:24:02.972 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T09:24:03.301 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T09:24:03.317 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T09:24:03.359 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T09:24:03.678 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T09:24:03.699 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T09:24:03.736 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T09:24:04.031 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T09:24:04.052 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T09:24:04.092 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T09:24:04.403 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T09:24:04.421 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T09:24:04.461 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T09:24:04.727 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T09:24:04.745 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T09:24:04.785 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T09:24:05.155 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T09:24:05.171 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T09:24:05.241 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package radosgw. 2024-09-13T09:24:05.257 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../110-radosgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:24:05.263 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T09:24:05.290 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:05.592 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T09:24:05.609 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T09:24:05.649 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T09:24:06.005 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T09:24:06.022 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T09:24:06.055 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T09:24:06.542 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T09:24:06.558 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T09:24:06.600 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T09:24:06.694 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T09:24:06.710 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:24:06.751 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:06.904 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T09:24:06.924 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T09:24:06.961 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T09:24:07.057 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T09:24:07.315 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T09:24:07.331 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T09:24:07.389 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T09:24:07.406 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T09:24:07.526 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T09:24:07.653 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T09:24:07.751 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T09:24:07.767 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T09:24:07.808 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T09:24:07.845 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:08.114 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T09:24:08.133 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T09:24:08.172 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T09:24:08.389 INFO:teuthology.orchestra.run.smithi063.stdout:Adding system user cephadm....done 2024-09-13T09:24:08.552 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:08.678 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T09:24:08.984 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package radosgw. 2024-09-13T09:24:09.001 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../110-radosgw_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:24:09.039 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T09:24:09.042 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:09.333 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:09.467 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T09:24:09.593 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T09:24:09.936 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T09:24:10.347 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T09:24:10.487 INFO:teuthology.orchestra.run.smithi099.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T09:24:10.505 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-820-g48a7de7e-1focal_amd64.deb ... 2024-09-13T09:24:10.570 INFO:teuthology.orchestra.run.smithi099.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:10.648 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T09:24:10.895 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T09:24:10.933 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T09:24:11.177 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T09:24:11.234 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T09:24:11.311 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T09:24:11.437 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T09:24:11.543 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:11.655 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:11.853 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T09:24:11.979 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T09:24:12.223 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user cephadm....done 2024-09-13T09:24:12.271 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T09:24:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T09:24:12.572 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T09:24:12.717 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T09:24:12.856 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T09:24:13.108 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T09:24:13.132 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T09:24:13.258 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T09:24:13.409 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T09:24:13.417 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T09:24:13.702 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T09:24:13.744 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T09:24:14.062 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T09:24:14.146 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T09:24:14.363 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T09:24:14.422 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T09:24:14.682 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T09:24:14.714 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T09:24:14.966 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T09:24:15.015 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T09:24:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:15.627 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T09:24:15.675 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T09:24:15.753 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T09:24:15.809 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T09:24:15.935 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T09:24:16.063 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T09:24:16.236 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T09:24:16.356 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T09:24:16.490 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T09:24:16.662 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T09:24:16.858 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T09:24:17.039 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T09:24:17.151 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T09:24:17.416 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T09:24:17.427 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T09:24:17.550 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:17.770 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T09:24:17.877 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:24:18.011 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T09:24:18.054 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T09:24:18.372 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T09:24:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T09:24:18.505 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T09:24:18.606 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T09:24:18.640 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T09:24:18.882 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T09:24:18.941 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T09:24:19.067 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T09:24:19.192 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:24:19.227 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T09:24:19.309 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T09:24:19.427 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T09:24:19.520 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T09:24:19.762 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T09:24:19.913 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T09:24:20.030 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T09:24:20.322 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T09:24:20.364 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T09:24:20.448 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:24:20.724 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T09:24:20.733 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T09:24:20.850 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:21.034 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T09:24:21.244 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:24:21.327 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T09:24:21.370 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T09:24:21.714 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T09:24:21.721 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T09:24:21.897 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T09:24:21.954 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T09:24:22.023 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T09:24:22.080 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T09:24:22.324 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T09:24:22.423 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T09:24:22.651 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:24:22.808 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T09:24:22.818 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T09:24:22.934 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T09:24:22.944 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T09:24:23.052 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T09:24:23.229 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T09:24:23.395 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T09:24:23.530 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T09:24:23.696 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T09:24:23.823 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T09:24:23.974 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:24:24.275 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T09:24:24.579 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T09:24:24.894 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T09:24:25.252 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:24:25.272 INFO:teuthology.orchestra.run.smithi099.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T09:24:25.377 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T09:24:25.555 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T09:24:25.701 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T09:24:25.754 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T09:24:26.063 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T09:24:26.365 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:24:26.441 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T09:24:26.491 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T09:24:26.850 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T09:24:26.851 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T09:24:26.968 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T09:24:27.102 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T09:24:27.185 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T09:24:27.468 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T09:24:27.570 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T09:24:27.755 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T09:24:27.896 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T09:24:28.960 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T09:24:29.086 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T09:24:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:24:29.244 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:24:29.338 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T09:24:29.403 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T09:24:29.656 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T09:24:29.780 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T09:24:29.782 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T09:24:30.081 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T09:24:30.100 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T09:24:30.382 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T09:24:30.384 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T09:24:30.533 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T09:24:30.703 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T09:24:30.893 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T09:24:31.063 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:31.189 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T09:24:31.203 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T09:24:31.315 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:31.441 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:31.571 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T09:24:31.644 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:31.770 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T09:24:31.905 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T09:24:32.080 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:32.206 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T09:24:32.508 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:32.643 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:32.769 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:32.969 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T09:24:32.982 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:24:32.982 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:24:33.095 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T09:24:33.254 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T09:24:33.381 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T09:24:33.448 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:33.572 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:33.673 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T09:24:33.697 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T09:24:33.799 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T09:24:34.151 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T09:24:34.444 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T09:24:34.652 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:34.745 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T09:24:34.777 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:34.887 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.013 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.114 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.139 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.240 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T09:24:35.399 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.525 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.709 INFO:teuthology.orchestra.run.smithi063.stdout:Adding group ceph....done 2024-09-13T09:24:35.762 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:35.888 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T09:24:36.086 INFO:teuthology.orchestra.run.smithi063.stdout:Adding system user ceph....done 2024-09-13T09:24:36.231 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:36.253 INFO:teuthology.orchestra.run.smithi063.stdout:Setting system user ceph properties....done 2024-09-13T09:24:36.272 INFO:teuthology.orchestra.run.smithi063.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T09:24:36.407 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T09:24:36.427 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T09:24:36.709 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:36.760 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T09:24:36.835 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:36.970 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:37.186 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:24:37.187 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:24:37.207 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:37.333 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:37.646 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:37.772 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:37.805 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:24:37.805 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:24:37.923 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T09:24:38.244 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:38.632 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T09:24:38.838 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:38.964 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:39.070 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:39.123 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:39.249 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:39.311 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:24:39.311 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:24:39.400 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:39.772 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:40.005 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:24:40.006 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:24:40.011 INFO:teuthology.orchestra.run.smithi099.stdout:Adding group ceph....done 2024-09-13T09:24:40.388 INFO:teuthology.orchestra.run.smithi099.stdout:Adding system user ceph....done 2024-09-13T09:24:40.457 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:40.556 INFO:teuthology.orchestra.run.smithi099.stdout:Setting system user ceph properties....done 2024-09-13T09:24:40.578 INFO:teuthology.orchestra.run.smithi099.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T09:24:40.732 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T09:24:40.952 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:24:40.952 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:24:41.051 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T09:24:41.468 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:41.544 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:41.594 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:41.653 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:41.737 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:41.967 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:24:41.967 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:24:42.113 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:24:42.116 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:24:42.421 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:42.547 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:42.556 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:42.665 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:42.808 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:42.960 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T09:24:43.186 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T09:24:43.338 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T09:24:43.416 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:43.623 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T09:24:43.624 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:24:43.625 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:24:44.043 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:44.282 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:24:44.282 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:24:44.728 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:45.204 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:24:45.204 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:24:45.662 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:45.788 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:45.964 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:46.193 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:24:46.193 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:24:46.633 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:46.801 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:46.911 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:46.959 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T09:24:47.052 INFO:teuthology.orchestra.run.smithi099.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1focal) ... 2024-09-13T09:24:47.407 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T09:24:47.558 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T09:24:47.738 INFO:teuthology.orchestra.run.smithi063.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T09:24:47.743 DEBUG:teuthology.parallel:result is None 2024-09-13T09:24:47.851 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T09:24:50.673 INFO:teuthology.orchestra.run.smithi099.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T09:24:51.313 INFO:teuthology.orchestra.run.smithi099.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T09:24:51.317 DEBUG:teuthology.parallel:result is None 2024-09-13T09:24:51.318 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:24:51.459 DEBUG:teuthology.orchestra.run.smithi063:> dpkg-query -W -f '${Version}' ceph 2024-09-13T09:24:51.483 INFO:teuthology.orchestra.run.smithi063.stdout:18.2.4-820-g48a7de7e-1focal 2024-09-13T09:24:51.483 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1focal 2024-09-13T09:24:51.483 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1focal is installed. 2024-09-13T09:24:51.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:24:51.627 DEBUG:teuthology.orchestra.run.smithi099:> dpkg-query -W -f '${Version}' ceph 2024-09-13T09:24:51.650 INFO:teuthology.orchestra.run.smithi099.stdout:18.2.4-820-g48a7de7e-1focal 2024-09-13T09:24:51.651 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1focal 2024-09-13T09:24:51.651 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1focal is installed. 2024-09-13T09:24:51.653 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T09:24:51.653 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:51.653 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:24:51.670 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:51.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:24:51.709 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T09:24:51.710 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:51.710 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:24:51.725 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:24:51.784 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:51.784 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:24:51.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:24:51.811 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T09:24:51.812 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:51.812 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:24:51.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:24:51.889 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:51.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:24:51.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:24:51.914 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T09:24:51.915 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:51.915 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:24:51.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:24:51.988 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:51.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:24:52.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:24:52.060 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T09:24:52.145 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T09:24:52.145 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-09-13T09:24:52.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-09-13T09:24:52.164 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T09:24:52.164 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:24:52.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:24:52.218 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T09:24:52.218 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:24:52.220 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:24:52.244 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T09:24:52.244 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T09:24:52.244 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901036', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi063.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T09:24:52.245 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:24:52.247 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:24:52.254 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:24:52.295 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:24:52.313 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:52.313 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:24:52.319 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:24:52.319 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-09-13T09:24:52.327 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:24:52.327 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.327 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 370 Links: 1 2024-09-13T09:24:52.327 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.328 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-13 09:21:19.643384757 +0000 2024-09-13T09:24:52.328 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-13 09:21:19.639384849 +0000 2024-09-13T09:24:52.328 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-13 09:21:19.639384849 +0000 2024-09-13T09:24:52.328 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-09-13T09:24:52.328 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:24:52.382 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-13T09:24:52.382 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-13T09:24:52.382 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000440483 s, 1.2 MB/s 2024-09-13T09:24:52.383 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:24:52.391 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-09-13T09:24:52.398 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:24:52.398 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.399 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 375 Links: 1 2024-09-13T09:24:52.399 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.399 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-13 09:21:20.275370189 +0000 2024-09-13T09:24:52.399 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-13 09:21:20.059375168 +0000 2024-09-13T09:24:52.399 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-13 09:21:20.059375168 +0000 2024-09-13T09:24:52.399 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-09-13T09:24:52.399 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:24:52.455 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-13T09:24:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-13T09:24:52.456 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000402071 s, 1.3 MB/s 2024-09-13T09:24:52.457 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:24:52.464 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 383 Links: 1 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-13 09:21:20.735359587 +0000 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-13 09:21:20.519364566 +0000 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-13 09:21:20.519364566 +0000 2024-09-13T09:24:52.471 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-09-13T09:24:52.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:24:52.527 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-13T09:24:52.527 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-13T09:24:52.527 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000423737 s, 1.2 MB/s 2024-09-13T09:24:52.529 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:24:52.537 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-09-13T09:24:52.542 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 6h/6d Inode: 388 Links: 1 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-09-13 09:21:21.167349629 +0000 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-09-13 09:21:20.959354423 +0000 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-09-13 09:21:20.959354423 +0000 2024-09-13T09:24:52.543 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-09-13T09:24:52.543 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:24:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-09-13T09:24:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-09-13T09:24:52.600 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000365235 s, 1.4 MB/s 2024-09-13T09:24:52.601 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:24:52.609 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:24:52.610 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:52.610 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:24:52.616 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:24:52.617 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-13T09:24:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:24:52.624 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.624 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 356 Links: 1 2024-09-13T09:24:52.624 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.625 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-13 09:21:18.987856165 +0000 2024-09-13T09:24:52.625 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-13 09:21:18.767861236 +0000 2024-09-13T09:24:52.625 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-13 09:21:18.767861236 +0000 2024-09-13T09:24:52.625 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-13T09:24:52.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:24:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-13T09:24:52.677 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-13T09:24:52.678 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000444 s, 1.2 MB/s 2024-09-13T09:24:52.678 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:24:52.687 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-13 09:21:19.219850818 +0000 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-13 09:21:19.215850910 +0000 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-13 09:21:19.215850910 +0000 2024-09-13T09:24:52.693 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-13T09:24:52.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:24:52.747 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-13T09:24:52.747 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-13T09:24:52.747 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000398848 s, 1.3 MB/s 2024-09-13T09:24:52.749 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:24:52.758 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 369 Links: 1 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-13 09:21:19.827836804 +0000 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-13 09:21:19.627841413 +0000 2024-09-13T09:24:52.765 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-13 09:21:19.627841413 +0000 2024-09-13T09:24:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-13T09:24:52.766 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:24:52.780 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-13T09:24:52.780 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-13T09:24:52.780 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000368554 s, 1.4 MB/s 2024-09-13T09:24:52.781 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:24:52.833 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-13 09:21:20.279826385 +0000 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-13 09:21:20.075831087 +0000 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-13 09:21:20.075831087 +0000 2024-09-13T09:24:52.840 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: - 2024-09-13T09:24:52.841 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:24:52.852 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-13T09:24:52.852 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-13T09:24:52.852 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000366588 s, 1.4 MB/s 2024-09-13T09:24:52.854 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:24:52.906 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:24:52.906 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T09:24:52.906 INFO:tasks.ceph:Generating config... 2024-09-13T09:24:52.907 INFO:tasks.ceph:[client] rbd cache = True 2024-09-13T09:24:52.907 INFO:tasks.ceph:[client] rbd clone copy on read = True 2024-09-13T09:24:52.907 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T09:24:52.908 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T09:24:52.908 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T09:24:52.908 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T09:24:52.908 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T09:24:52.908 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T09:24:52.908 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T09:24:52.908 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T09:24:52.908 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T09:24:52.908 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-13T09:24:52.908 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-13T09:24:52.908 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T09:24:52.908 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T09:24:52.908 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T09:24:52.909 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T09:24:52.909 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T09:24:52.910 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T09:24:52.951 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T09:24:52.963 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T09:24:53.048 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:24:53.062 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.63'), ('mon.c', '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]'), ('mon.b', '172.21.15.99')] 2024-09-13T09:24:53.062 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.99', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd clone copy on read': True}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T09:24:53.063 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:53.063 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:24:53.070 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.63 --addv c '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]' --add b 172.21.15.99 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool: generated fsid eb7f03c3-9691-4766-8999-6f4f526cda71 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = pacific 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 0 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:fsid eb7f03c3-9691-4766-8999-6f4f526cda71 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:last_changed 2024-09-13T09:24:53.111073+0000 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-09-13T09:24:53.111073+0000 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:min_mon_release 16 (pacific) 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:election_strategy: 1 2024-09-13T09:24:53.113 INFO:teuthology.orchestra.run.smithi063.stdout:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-09-13T09:24:53.114 INFO:teuthology.orchestra.run.smithi063.stdout:1: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-13T09:24:53.114 INFO:teuthology.orchestra.run.smithi063.stdout:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-09-13T09:24:53.114 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T09:24:53.115 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:24:53.121 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID eb7f03c3-9691-4766-8999-6f4f526cda71... 2024-09-13T09:24:53.122 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:24:53.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2024-09-13T09:24:53.199 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.200 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:24:53.201 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.99" 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon client directed command retry = 5 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on pool no app = False 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: ms inject socket failures = 5000 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = eb7f03c3-9691-4766-8999-6f4f526cda71 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.202 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.203 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2024-09-13T09:24:53.204 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression algorithm = snappy 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression mode = aggressive 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:24:53.205 INFO:teuthology.orchestra.run.smithi099.stdout: osd max markdown count = 1000 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: osd shutdown pgref assert = True 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2024-09-13T09:24:53.206 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:24:53.207 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:24:53.208 INFO:teuthology.orchestra.run.smithi099.stdout: rbd cache = True 2024-09-13T09:24:53.209 INFO:teuthology.orchestra.run.smithi099.stdout: rbd clone copy on read = True 2024-09-13T09:24:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2024-09-13T09:24:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2024-09-13T09:24:53.209 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2024-09-13T09:24:53.210 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-09-13T09:24:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-09-13T09:24:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:24:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-09-13T09:24:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-09-13T09:24:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-09-13T09:24:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.99" 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon client directed command retry = 5 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on pool no app = False 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: ms inject socket failures = 5000 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = eb7f03c3-9691-4766-8999-6f4f526cda71 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-09-13T09:24:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore compression algorithm = snappy 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore compression mode = aggressive 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 1/20 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 1/20 2024-09-13T09:24:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 4/10 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: osd max markdown count = 1000 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: osd shutdown pgref assert = True 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-09-13T09:24:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:24:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:24:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: rbd cache = True 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: rbd clone copy on read = True 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-09-13T09:24:53.220 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-09-13T09:24:53.221 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T09:24:53.221 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T09:24:53.277 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T09:24:53.278 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:53.278 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T09:24:53.285 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:53.285 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T09:24:53.292 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:24:53.292 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:53.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:24:53.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:24:53.317 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:24:53.317 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:24:53.321 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:24:53.321 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:53.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:24:53.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:24:53.349 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:24:53.349 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:24:53.355 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T09:24:53.355 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T09:24:53.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T09:24:53.415 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T09:24:53.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:24:53.469 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:24:53.478 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T09:24:53.478 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T09:24:53.478 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T09:24:53.520 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T09:24:53.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T09:24:53.577 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T09:24:53.591 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T09:24:53.591 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T09:24:53.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T09:24:53.608 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:24:53.608 INFO:tasks.ceph:role: osd.0 2024-09-13T09:24:53.608 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:24:53.608 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:24:54.123 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:24:54.124 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:24:54.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-09-13T09:24:54.125 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T09:24:54.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T09:24:54.249 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:24:54.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:24:54.250 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T09:24:54.262 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:24:54.262 INFO:tasks.ceph:role: osd.1 2024-09-13T09:24:54.262 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:24:54.262 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:24:54.781 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:24:54.781 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:24:54.781 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:24:54.781 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 2024-09-13T09:24:54.781 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:24:54.782 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:24:54.782 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:24:54.782 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:24:54.782 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:24:54.782 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:24:54.783 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-09-13T09:24:54.783 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T09:24:54.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T09:24:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:24:54.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:24:54.840 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T09:24:54.850 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:24:54.850 INFO:tasks.ceph:role: osd.2 2024-09-13T09:24:54.850 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:24:54.850 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:24:55.302 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:24:55.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-09-13T09:24:55.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T09:24:55.348 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T09:24:55.358 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:24:55.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:24:55.359 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T09:24:55.417 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:24:55.417 INFO:tasks.ceph:role: osd.3 2024-09-13T09:24:55.417 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:24:55.417 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:24:55.925 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:24:55.926 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:24:55.926 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:24:55.927 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-09-13T09:24:55.927 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T09:24:55.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T09:24:56.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:24:56.003 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:24:56.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:24:56.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:56.228+0000 7fe4dab8b880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T09:24:56.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:56.228+0000 7fe4dab8b880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T09:24:56.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:56.228+0000 7fe4dab8b880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:24:56.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:56.228+0000 7fe4dab8b880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:24:56.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:56.228+0000 7fe4dab8b880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:24:56.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:56.236+0000 7fe4dab8b880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T09:24:59.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T09:24:59.259 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:24:59.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:59.292+0000 7f15a019b880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T09:24:59.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:59.292+0000 7f15a019b880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T09:24:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:59.292+0000 7f15a019b880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:24:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:59.292+0000 7f15a019b880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:24:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:59.292+0000 7f15a019b880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:24:59.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:24:59.296+0000 7f15a019b880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T09:25:02.293 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T09:25:02.353 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:02.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:02.388+0000 7fc39db13880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T09:25:02.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:02.388+0000 7fc39db13880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T09:25:02.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:02.388+0000 7fc39db13880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:25:02.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:02.388+0000 7fc39db13880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:25:02.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:02.388+0000 7fc39db13880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:25:02.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:02.388+0000 7fc39db13880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T09:25:05.388 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T09:25:05.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:05.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:05.492+0000 7fb18af14880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T09:25:05.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:05.492+0000 7fb18af14880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T09:25:05.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:05.492+0000 7fb18af14880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:25:05.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:05.492+0000 7fb18af14880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:25:05.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:05.492+0000 7fb18af14880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:25:05.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:25:05.492+0000 7fb18af14880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T09:25:08.480 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T09:25:08.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T09:25:08.513 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:25:08.513 INFO:tasks.ceph:role: osd.4 2024-09-13T09:25:08.514 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:25:08.514 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:25:09.043 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:25:09.043 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:25:09.043 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:25:09.043 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2024-09-13T09:25:09.043 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:25:09.043 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:25:09.044 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:25:09.044 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:25:09.044 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:25:09.044 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:25:09.045 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-13T09:25:09.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T09:25:09.161 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T09:25:09.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:25:09.174 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:25:09.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T09:25:09.188 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:25:09.189 INFO:tasks.ceph:role: osd.5 2024-09-13T09:25:09.189 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:25:09.189 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:25:09.708 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:25:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:25:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:25:09.708 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2024-09-13T09:25:09.709 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:25:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:25:09.709 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:25:09.709 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:25:09.709 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:25:09.709 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:25:09.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-13T09:25:09.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T09:25:09.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T09:25:09.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:25:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:25:09.755 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T09:25:09.765 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:25:09.765 INFO:tasks.ceph:role: osd.6 2024-09-13T09:25:09.765 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:25:09.765 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:25:10.295 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:25:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:25:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:25:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2024-09-13T09:25:10.295 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:25:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:25:10.296 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:25:10.296 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:25:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:25:10.296 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:25:10.297 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-13T09:25:10.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T09:25:10.358 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T09:25:10.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:25:10.370 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:25:10.371 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T09:25:10.427 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:25:10.427 INFO:tasks.ceph:role: osd.7 2024-09-13T09:25:10.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:25:10.427 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:25:10.951 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:25:10.952 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:25:10.952 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:25:10.953 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-13T09:25:10.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T09:25:11.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T09:25:11.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:25:11.031 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:25:11.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:11.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:11.158+0000 7f2d433a0880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T09:25:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:11.158+0000 7f2d433a0880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T09:25:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:11.158+0000 7f2d433a0880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:25:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:11.158+0000 7f2d433a0880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:25:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:11.158+0000 7f2d433a0880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:25:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:11.166+0000 7f2d433a0880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T09:25:14.185 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T09:25:14.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:14.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:14.234+0000 7f8a72634880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T09:25:14.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:14.234+0000 7f8a72634880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T09:25:14.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:14.234+0000 7f8a72634880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:25:14.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:14.234+0000 7f8a72634880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:25:14.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:14.234+0000 7f8a72634880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:25:14.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:14.238+0000 7f8a72634880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T09:25:17.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T09:25:17.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:17.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:17.338+0000 7f7ae6da1880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T09:25:17.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:17.338+0000 7f7ae6da1880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T09:25:17.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:17.338+0000 7f7ae6da1880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:25:17.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:17.338+0000 7f7ae6da1880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:25:17.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:17.338+0000 7f7ae6da1880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:25:17.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:17.338+0000 7f7ae6da1880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T09:25:20.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T09:25:20.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:20.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:20.382+0000 7f2f7fd9e880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T09:25:20.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:20.382+0000 7f2f7fd9e880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T09:25:20.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:20.382+0000 7f2f7fd9e880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:25:20.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:20.382+0000 7f2f7fd9e880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:25:20.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:20.382+0000 7f2f7fd9e880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:25:20.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:20.386+0000 7f2f7fd9e880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T09:25:23.397 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T09:25:23.457 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T09:25:23.457 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:23.457 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T09:25:23.474 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:23.474 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T09:25:23.533 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:23.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T09:25:23.544 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:23.544 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T09:25:23.556 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:23.556 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T09:25:23.615 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:23.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T09:25:23.631 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:23.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T09:25:23.643 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:23.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T09:25:23.653 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:23.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T09:25:23.665 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:23.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T09:25:23.725 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:23.725 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T09:25:23.732 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:23.732 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T09:25:23.738 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T09:25:23.738 DEBUG:teuthology.orchestra.run.smithi063:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:25:23.741 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.y] 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDlBORmXf6qGBAAZ0BIAUaWFZCOowLecwzuDA== 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.0] 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDoBORmOmP4DRAAA0VwHJTNzkwXXHLHFi8c7A== 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.1] 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDrBORmq4nRERAABv2m++p+Bpv7DxOTI/jXHg== 2024-09-13T09:25:23.784 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.2] 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDuBORmOpx8FxAAj6InC1xb3l5W6CQJngqqwg== 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.3] 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDxBORmcEeVHRAA6Djn8BMq/2lEUG2pIiPHmA== 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.x] 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDlBORmA5bvGxAAeowmhK8RFmTur1Gdmcqmyw== 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.4] 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQD3BORm06GlCRAAojlFfIpTcbU4yhcRkBUdGw== 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.5] 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQD6BORm60lPDhAAcBxLkA88PXyRINGz0imgMQ== 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.6] 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQD9BORmqDZfFBAApRyQxnj1Eqm+riMFQCXqfQ== 2024-09-13T09:25:23.785 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.7] 2024-09-13T09:25:23.786 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQAABeRmUhspFxAABuocgV4WPOOQ1yG9XB7J1Q== 2024-09-13T09:25:23.786 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-09-13T09:25:23.786 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDlBORm4G3tHhAACM1Lm+4PgduzFTOp6UzdNA== 2024-09-13T09:25:23.786 INFO:teuthology.orchestra.run.smithi063.stdout:[client.1] 2024-09-13T09:25:23.786 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQDlBORmDG9pIhAAQQqtSvfRrcHLqZe6uEhZ0w== 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDlBORmXf6qGBAAZ0BIAUaWFZCOowLecwzuDA== 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDoBORmOmP4DRAAA0VwHJTNzkwXXHLHFi8c7A== 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDrBORmq4nRERAABv2m++p+Bpv7DxOTI/jXHg== 2024-09-13T09:25:23.795 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDuBORmOpx8FxAAj6InC1xb3l5W6CQJngqqwg== 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDxBORmcEeVHRAA6Djn8BMq/2lEUG2pIiPHmA== 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDlBORmA5bvGxAAeowmhK8RFmTur1Gdmcqmyw== 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD3BORm06GlCRAAojlFfIpTcbU4yhcRkBUdGw== 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD6BORm60lPDhAAcBxLkA88PXyRINGz0imgMQ== 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD9BORmqDZfFBAApRyQxnj1Eqm+riMFQCXqfQ== 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2024-09-13T09:25:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQAABeRmUhspFxAABuocgV4WPOOQ1yG9XB7J1Q== 2024-09-13T09:25:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-13T09:25:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDlBORm4G3tHhAACM1Lm+4PgduzFTOp6UzdNA== 2024-09-13T09:25:23.797 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-09-13T09:25:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDlBORmDG9pIhAAQQqtSvfRrcHLqZe6uEhZ0w== 2024-09-13T09:25:23.798 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:25:23.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:25:23.883 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:23.886 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:23.935 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:23.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:23.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:23.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.037 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.089 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:25:24.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:25:24.137 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.233 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.282 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.375 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:25:24.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:25:24.429 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:25:24.481 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:25:24.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:25:24.532 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T09:25:24.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T09:25:24.546 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:25:24.680 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T09:25:24.695 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T09:25:24.707 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:25:24.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T09:25:24.812 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T09:25:24.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:25:24.967 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T09:25:24.984 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:24.986 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:25:24.991 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T09:25:24.992 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T09:25:24.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T09:25:25.035 INFO:tasks.ceph.mon.a:Started 2024-09-13T09:25:25.035 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T09:25:25.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T09:25:25.038 INFO:tasks.ceph.mon.c:Started 2024-09-13T09:25:25.038 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T09:25:25.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T09:25:25.040 INFO:tasks.ceph.mon.b:Started 2024-09-13T09:25:25.040 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T09:25:25.041 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T09:25:25.041 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T09:25:25.043 INFO:tasks.ceph.mgr.y:Started 2024-09-13T09:25:25.043 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T09:25:25.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T09:25:25.045 INFO:tasks.ceph.mgr.x:Started 2024-09-13T09:25:25.046 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T09:25:25.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T09:25:30.506 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-09-13T09:25:30.519 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T09:25:30.519 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T09:25:30.520 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:30.520 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T09:25:30.530 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:30.530 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T09:25:30.545 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:30.545 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T09:25:30.555 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-09-13T09:25:30.555 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T09:25:30.570 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:30.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T09:25:30.585 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:30.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T09:25:30.644 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:30.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T09:25:30.655 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-13T09:25:30.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T09:25:30.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 54ea2bd8-55b0-4ded-8a16-041261c5ba5a 0 2024-09-13T09:25:30.891 INFO:teuthology.orchestra.run.smithi099.stdout:0 2024-09-13T09:25:30.906 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 0c95aee9-f871-4f6f-bbbd-1206db5d331c 1 2024-09-13T09:25:31.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-13T09:25:31.230+0000 7f3030834700 0 --2- 172.21.15.99:0/3812771692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f302c13ec00 0x7f302c1d55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:25:31.334 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-13T09:25:31.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new f8253db2-d4da-46b5-95cf-cd5c9ebd7328 2 2024-09-13T09:25:31.383 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:31.378+0000 7f3630dcb2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:25:31.385 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:31.375+0000 7f671aebf2c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:25:31.544 INFO:teuthology.orchestra.run.smithi099.stdout:2 2024-09-13T09:25:31.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 67c11a52-baeb-4ea8-89ea-26018ffdc35a 3 2024-09-13T09:25:31.760 INFO:teuthology.orchestra.run.smithi099.stdout:3 2024-09-13T09:25:31.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 1cec9702-fa09-4d7c-b111-7d10b1950cfe 4 2024-09-13T09:25:31.991 INFO:teuthology.orchestra.run.smithi099.stdout:4 2024-09-13T09:25:32.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 7c0b908a-65c9-426c-867c-cc8ed65084b3 5 2024-09-13T09:25:32.135 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.127+0000 7f671aebf2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:25:32.180 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.174+0000 7f3630dcb2c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:25:32.195 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.187+0000 7f671aebf2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:25:32.215 INFO:teuthology.orchestra.run.smithi099.stdout:5 2024-09-13T09:25:32.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 78d5990a-4535-4110-ba04-88f01ab9fe28 6 2024-09-13T09:25:32.239 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.234+0000 7f3630dcb2c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:25:32.372 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.363+0000 7f671aebf2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:25:32.416 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.410+0000 7f3630dcb2c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:25:32.431 INFO:teuthology.orchestra.run.smithi099.stdout:6 2024-09-13T09:25:32.444 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 53f8511c-729f-4423-8eac-813c01f8f72b 7 2024-09-13T09:25:32.609 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.599+0000 7f671aebf2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:25:32.638 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.634+0000 7f3630dcb2c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:25:32.642 INFO:teuthology.orchestra.run.smithi099.stdout:7 2024-09-13T09:25:32.655 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T09:25:32.655 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T09:25:32.657 INFO:tasks.ceph.osd.0:Started 2024-09-13T09:25:32.657 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T09:25:32.657 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T09:25:32.660 INFO:tasks.ceph.osd.1:Started 2024-09-13T09:25:32.660 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T09:25:32.660 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T09:25:32.662 INFO:tasks.ceph.osd.2:Started 2024-09-13T09:25:32.662 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T09:25:32.662 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T09:25:32.665 INFO:tasks.ceph.osd.3:Started 2024-09-13T09:25:32.665 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T09:25:32.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T09:25:32.667 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.659+0000 7f671aebf2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:25:32.667 INFO:tasks.ceph.osd.4:Started 2024-09-13T09:25:32.667 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T09:25:32.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T09:25:32.669 INFO:tasks.ceph.osd.5:Started 2024-09-13T09:25:32.670 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T09:25:32.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T09:25:32.672 INFO:tasks.ceph.osd.6:Started 2024-09-13T09:25:32.672 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T09:25:32.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T09:25:32.674 INFO:tasks.ceph.osd.7:Started 2024-09-13T09:25:32.675 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:25:32.703 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.698+0000 7f3630dcb2c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:25:32.877 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:32.877 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":10,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:25:32.632800+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:25:32.632800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:25:32.889 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.886+0000 7f3630dcb2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:25:32.890 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T09:25:32.891 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T09:25:32.891 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.883+0000 7f671aebf2c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:25:32.953 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:32.946+0000 7f3630dcb2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:25:32.956 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:32.947+0000 7f671aebf2c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:25:33.017 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:33.014+0000 7f3630dcb2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:25:33.019 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:33.014+0000 7f0b7cffa880 -1 Falling back to public interface 2024-09-13T09:25:33.021 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:33.011+0000 7f671aebf2c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:25:33.039 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:33.031+0000 7fd69857a880 -1 Falling back to public interface 2024-09-13T09:25:33.059 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:33.051+0000 7f1618d16880 -1 Falling back to public interface 2024-09-13T09:25:33.283 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:33.278+0000 7ff52a3c7880 -1 Falling back to public interface 2024-09-13T09:25:33.283 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:33.278+0000 7fe64e681880 -1 Falling back to public interface 2024-09-13T09:25:33.295 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:33.290+0000 7fd02122a880 -1 Falling back to public interface 2024-09-13T09:25:33.304 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:33.295+0000 7f4e54a31880 -1 Falling back to public interface 2024-09-13T09:25:33.315 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:33.307+0000 7f42c0736880 -1 Falling back to public interface 2024-09-13T09:25:33.410 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:33.406+0000 7f3630dcb2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:25:33.446 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:33.435+0000 7f671aebf2c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:25:33.527 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:33.522+0000 7f3630dcb2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:25:33.566 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:33.559+0000 7f671aebf2c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:25:33.695 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:25:33.856 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:33.850+0000 7f3630dcb2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:25:33.874 INFO:teuthology.misc.health.smithi063.stdout: 2024-09-13T09:25:33.874 INFO:teuthology.misc.health.smithi063.stdout:{"epoch":10,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:25:32.632800+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:25:32.632800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:25:33.885 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T09:25:33.917 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:33.907+0000 7f671aebf2c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:25:34.044 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:34.038+0000 7f3630dcb2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:25:34.108 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:34.099+0000 7f671aebf2c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:25:34.112 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:34.106+0000 7f3630dcb2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:25:34.179 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:34.171+0000 7f671aebf2c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:25:34.271 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:34.266+0000 7f3630dcb2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:25:34.342 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:34.335+0000 7f671aebf2c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:25:34.988 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:34.979+0000 7f671aebf2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:25:35.009 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.002+0000 7f3630dcb2c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:25:35.060 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:35.051+0000 7f671aebf2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:25:35.079 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.074+0000 7f3630dcb2c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:25:35.232 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.226+0000 7f3630dcb2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:25:35.241 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:35.231+0000 7f671aebf2c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:25:35.300 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.294+0000 7f3630dcb2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:25:35.312 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:35.303+0000 7f671aebf2c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:25:35.374 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.370+0000 7f3630dcb2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:25:35.385 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:35.375+0000 7f671aebf2c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:25:35.697 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.690+0000 7f3630dcb2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:25:35.723 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:35.715+0000 7f671aebf2c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:25:35.769 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:35.762+0000 7f3630dcb2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:25:35.802 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:35.795+0000 7f671aebf2c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:25:36.482 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:36.478+0000 7f3630dcb2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:25:36.485 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:36.475+0000 7fd69857a880 -1 osd.2 0 log_to_monitors true 2024-09-13T09:25:36.540 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-09-13T09:25:36.531+0000 7f671aebf2c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:25:36.628 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:36.622+0000 7fe64e681880 -1 osd.6 0 log_to_monitors true 2024-09-13T09:25:36.840 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:36.834+0000 7f0b7cffa880 -1 osd.4 0 log_to_monitors true 2024-09-13T09:25:36.895 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:36.890+0000 7fd02122a880 -1 osd.7 0 log_to_monitors true 2024-09-13T09:25:36.940 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:36.934+0000 7ff52a3c7880 -1 osd.5 0 log_to_monitors true 2024-09-13T09:25:36.999 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:36.991+0000 7f1618d16880 -1 osd.3 0 log_to_monitors true 2024-09-13T09:25:37.186 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:37.179+0000 7f42c0736880 -1 osd.1 0 log_to_monitors true 2024-09-13T09:25:37.221 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:37.211+0000 7f4e54a31880 -1 osd.0 0 log_to_monitors true 2024-09-13T09:25:37.529 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:37.526+0000 7f3617942700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T09:25:37.530 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-13T09:25:37.526+0000 7f3617942700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T09:25:38.851 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:38.843+0000 7fd6943b9700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T09:25:38.861 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:38.851+0000 7fd68be68700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:39.911 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:39.905+0000 7f0b797c0700 -1 osd.4 0 waiting for initial osdmap 2024-09-13T09:25:39.914 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:39.907+0000 7f16154dc700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T09:25:39.919 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:39.913+0000 7f0b708e8700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:39.922 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:39.911+0000 7f160c604700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:39.957 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:39.947+0000 7f42bc575700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T09:25:39.959 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:39.953+0000 7fd01d069700 -1 osd.7 0 waiting for initial osdmap 2024-09-13T09:25:39.962 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:39.955+0000 7f42b4024700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:39.971 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:39.965+0000 7fd014b18700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:39.978 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:39.971+0000 7f4e50870700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T09:25:39.982 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:39.979+0000 7f4e4831f700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:39.994 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:39.989+0000 7fe64a4c0700 -1 osd.6 0 waiting for initial osdmap 2024-09-13T09:25:39.998 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:39.993+0000 7fe641f6f700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:40.012 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:40.005+0000 7ff526206700 -1 osd.5 0 waiting for initial osdmap 2024-09-13T09:25:40.016 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:40.009+0000 7ff51dcb5700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:25:40.692 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:25:41.003 INFO:teuthology.misc.health.smithi063.stdout: 2024-09-13T09:25:41.003 INFO:teuthology.misc.health.smithi063.stdout:{"epoch":15,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:25:40.920760+0000","last_up_change":"2024-09-13T09:25:40.920760+0000","last_in_change":"2024-09-13T09:25:32.632800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6817","nonce":2906107514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6819","nonce":2906107514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6823","nonce":2906107514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6821","nonce":2906107514}]},"public_addr":"172.21.15.63:6817/2906107514","cluster_addr":"172.21.15.63:6819/2906107514","heartbeat_back_addr":"172.21.15.63:6823/2906107514","heartbeat_front_addr":"172.21.15.63:6821/2906107514","state":["exists","up"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6825","nonce":3622854518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6827","nonce":3622854518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6831","nonce":3622854518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6829","nonce":3622854518}]},"public_addr":"172.21.15.63:6825/3622854518","cluster_addr":"172.21.15.63:6827/3622854518","heartbeat_back_addr":"172.21.15.63:6831/3622854518","heartbeat_front_addr":"172.21.15.63:6829/3622854518","state":["exists","up"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6801","nonce":2105659441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6803","nonce":2105659441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6807","nonce":2105659441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6805","nonce":2105659441}]},"public_addr":"172.21.15.63:6801/2105659441","cluster_addr":"172.21.15.63:6803/2105659441","heartbeat_back_addr":"172.21.15.63:6807/2105659441","heartbeat_front_addr":"172.21.15.63:6805/2105659441","state":["exists","up"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6809","nonce":4165134522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6811","nonce":4165134522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6815","nonce":4165134522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6813","nonce":4165134522}]},"public_addr":"172.21.15.63:6809/4165134522","cluster_addr":"172.21.15.63:6811/4165134522","heartbeat_back_addr":"172.21.15.63:6815/4165134522","heartbeat_front_addr":"172.21.15.63:6813/4165134522","state":["exists","up"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6801","nonce":1789187713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6803","nonce":1789187713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6807","nonce":1789187713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6805","nonce":1789187713}]},"public_addr":"172.21.15.99:6801/1789187713","cluster_addr":"172.21.15.99:6803/1789187713","heartbeat_back_addr":"172.21.15.99:6807/1789187713","heartbeat_front_addr":"172.21.15.99:6805/1789187713","state":["exists","up"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6809","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6811","nonce":3410135021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6813","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6815","nonce":3410135021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6823","nonce":3410135021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6819","nonce":3410135021}]},"public_addr":"172.21.15.99:6811/3410135021","cluster_addr":"172.21.15.99:6815/3410135021","heartbeat_back_addr":"172.21.15.99:6823/3410135021","heartbeat_front_addr":"172.21.15.99:6819/3410135021","state":["exists","up"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6810","nonce":4152410871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6814","nonce":4152410871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6822","nonce":4152410871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6818","nonce":4152410871}]},"public_addr":"172.21.15.99:6810/4152410871","cluster_addr":"172.21.15.99:6814/4152410871","heartbeat_back_addr":"172.21.15.99:6822/4152410871","heartbeat_front_addr":"172.21.15.99:6818/4152410871","state":["exists","up"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6825","nonce":2923799929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6827","nonce":2923799929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6831","nonce":2923799929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6829","nonce":2923799929}]},"public_addr":"172.21.15.99:6825/2923799929","cluster_addr":"172.21.15.99:6827/2923799929","heartbeat_back_addr":"172.21.15.99:6831/2923799929","heartbeat_front_addr":"172.21.15.99:6829/2923799929","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.447964+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:25:41.016 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T09:25:41.016 INFO:tasks.ceph:Creating RBD pool 2024-09-13T09:25:41.016 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T09:25:41.942 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'rbd' created 2024-09-13T09:25:41.954 DEBUG:teuthology.orchestra.run.smithi063:> rbd --cluster ceph pool init rbd 2024-09-13T09:25:44.977 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T09:25:44.977 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T09:25:44.978 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T09:25:44.978 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T09:25:44.978 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T09:25:45.335 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:45.350 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":4,"flags":0,"active_gid":4123,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1740862197},{"type":"v1","addr":"172.21.15.99:6833","nonce":1740862197}]},"active_addr":"172.21.15.99:6833/1740862197","active_change":"2024-09-13T09:25:36.501564+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1560218502}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3429866558}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1116654084}]}]} 2024-09-13T09:25:45.353 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T09:25:45.354 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:25:45.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:25:45.656 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:45.657 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:25:44.948342+0000","last_up_change":"2024-09-13T09:25:40.920760+0000","last_in_change":"2024-09-13T09:25:32.632800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T09:25:41.357433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T09:25:42.578616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6817","nonce":2906107514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6819","nonce":2906107514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6823","nonce":2906107514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6821","nonce":2906107514}]},"public_addr":"172.21.15.63:6817/2906107514","cluster_addr":"172.21.15.63:6819/2906107514","heartbeat_back_addr":"172.21.15.63:6823/2906107514","heartbeat_front_addr":"172.21.15.63:6821/2906107514","state":["exists","up"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6825","nonce":3622854518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6827","nonce":3622854518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6831","nonce":3622854518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6829","nonce":3622854518}]},"public_addr":"172.21.15.63:6825/3622854518","cluster_addr":"172.21.15.63:6827/3622854518","heartbeat_back_addr":"172.21.15.63:6831/3622854518","heartbeat_front_addr":"172.21.15.63:6829/3622854518","state":["exists","up"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6801","nonce":2105659441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6803","nonce":2105659441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6807","nonce":2105659441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6805","nonce":2105659441}]},"public_addr":"172.21.15.63:6801/2105659441","cluster_addr":"172.21.15.63:6803/2105659441","heartbeat_back_addr":"172.21.15.63:6807/2105659441","heartbeat_front_addr":"172.21.15.63:6805/2105659441","state":["exists","up"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6809","nonce":4165134522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6811","nonce":4165134522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6815","nonce":4165134522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6813","nonce":4165134522}]},"public_addr":"172.21.15.63:6809/4165134522","cluster_addr":"172.21.15.63:6811/4165134522","heartbeat_back_addr":"172.21.15.63:6815/4165134522","heartbeat_front_addr":"172.21.15.63:6813/4165134522","state":["exists","up"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6801","nonce":1789187713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6803","nonce":1789187713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6807","nonce":1789187713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6805","nonce":1789187713}]},"public_addr":"172.21.15.99:6801/1789187713","cluster_addr":"172.21.15.99:6803/1789187713","heartbeat_back_addr":"172.21.15.99:6807/1789187713","heartbeat_front_addr":"172.21.15.99:6805/1789187713","state":["exists","up"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6809","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6811","nonce":3410135021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6813","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6815","nonce":3410135021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6823","nonce":3410135021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6819","nonce":3410135021}]},"public_addr":"172.21.15.99:6811/3410135021","cluster_addr":"172.21.15.99:6815/3410135021","heartbeat_back_addr":"172.21.15.99:6823/3410135021","heartbeat_front_addr":"172.21.15.99:6819/3410135021","state":["exists","up"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6810","nonce":4152410871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6814","nonce":4152410871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6822","nonce":4152410871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6818","nonce":4152410871}]},"public_addr":"172.21.15.99:6810/4152410871","cluster_addr":"172.21.15.99:6814/4152410871","heartbeat_back_addr":"172.21.15.99:6822/4152410871","heartbeat_front_addr":"172.21.15.99:6818/4152410871","state":["exists","up"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6825","nonce":2923799929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6827","nonce":2923799929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6831","nonce":2923799929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6829","nonce":2923799929}]},"public_addr":"172.21.15.99:6825/2923799929","cluster_addr":"172.21.15.99:6827/2923799929","heartbeat_back_addr":"172.21.15.99:6831/2923799929","heartbeat_front_addr":"172.21.15.99:6829/2923799929","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.267895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.161313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.447964+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.021320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.883961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.938146+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.627237+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.887523+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:25:45.681 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:25:45.681 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:25:45.977 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:45.978 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:25:44.948342+0000","last_up_change":"2024-09-13T09:25:40.920760+0000","last_in_change":"2024-09-13T09:25:32.632800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T09:25:41.357433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T09:25:42.578616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6817","nonce":2906107514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6819","nonce":2906107514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6823","nonce":2906107514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6821","nonce":2906107514}]},"public_addr":"172.21.15.63:6817/2906107514","cluster_addr":"172.21.15.63:6819/2906107514","heartbeat_back_addr":"172.21.15.63:6823/2906107514","heartbeat_front_addr":"172.21.15.63:6821/2906107514","state":["exists","up"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6825","nonce":3622854518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6827","nonce":3622854518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6831","nonce":3622854518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6829","nonce":3622854518}]},"public_addr":"172.21.15.63:6825/3622854518","cluster_addr":"172.21.15.63:6827/3622854518","heartbeat_back_addr":"172.21.15.63:6831/3622854518","heartbeat_front_addr":"172.21.15.63:6829/3622854518","state":["exists","up"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6801","nonce":2105659441}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6803","nonce":2105659441}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6807","nonce":2105659441}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2105659441},{"type":"v1","addr":"172.21.15.63:6805","nonce":2105659441}]},"public_addr":"172.21.15.63:6801/2105659441","cluster_addr":"172.21.15.63:6803/2105659441","heartbeat_back_addr":"172.21.15.63:6807/2105659441","heartbeat_front_addr":"172.21.15.63:6805/2105659441","state":["exists","up"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6809","nonce":4165134522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6811","nonce":4165134522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6815","nonce":4165134522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":4165134522},{"type":"v1","addr":"172.21.15.63:6813","nonce":4165134522}]},"public_addr":"172.21.15.63:6809/4165134522","cluster_addr":"172.21.15.63:6811/4165134522","heartbeat_back_addr":"172.21.15.63:6815/4165134522","heartbeat_front_addr":"172.21.15.63:6813/4165134522","state":["exists","up"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6801","nonce":1789187713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6803","nonce":1789187713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6807","nonce":1789187713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":1789187713},{"type":"v1","addr":"172.21.15.99:6805","nonce":1789187713}]},"public_addr":"172.21.15.99:6801/1789187713","cluster_addr":"172.21.15.99:6803/1789187713","heartbeat_back_addr":"172.21.15.99:6807/1789187713","heartbeat_front_addr":"172.21.15.99:6805/1789187713","state":["exists","up"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6809","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6811","nonce":3410135021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6813","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6815","nonce":3410135021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6823","nonce":3410135021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":3410135021},{"type":"v1","addr":"172.21.15.99:6819","nonce":3410135021}]},"public_addr":"172.21.15.99:6811/3410135021","cluster_addr":"172.21.15.99:6815/3410135021","heartbeat_back_addr":"172.21.15.99:6823/3410135021","heartbeat_front_addr":"172.21.15.99:6819/3410135021","state":["exists","up"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6810","nonce":4152410871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6814","nonce":4152410871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6822","nonce":4152410871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6818","nonce":4152410871}]},"public_addr":"172.21.15.99:6810/4152410871","cluster_addr":"172.21.15.99:6814/4152410871","heartbeat_back_addr":"172.21.15.99:6822/4152410871","heartbeat_front_addr":"172.21.15.99:6818/4152410871","state":["exists","up"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6825","nonce":2923799929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6827","nonce":2923799929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6831","nonce":2923799929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6829","nonce":2923799929}]},"public_addr":"172.21.15.99:6825/2923799929","cluster_addr":"172.21.15.99:6827/2923799929","heartbeat_back_addr":"172.21.15.99:6831/2923799929","heartbeat_front_addr":"172.21.15.99:6829/2923799929","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.267895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.161313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.447964+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.021320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.883961+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.938146+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.627237+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.887523+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:25:45.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:25:45.993 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:25:45.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:25:45.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:25:45.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:25:45.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:25:45.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T09:25:45.996 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T09:25:46.207 INFO:teuthology.orchestra.run.smithi063.stdout:64424509444 2024-09-13T09:25:46.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:25:46.211 INFO:teuthology.orchestra.run.smithi063.stdout:64424509442 2024-09-13T09:25:46.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:25:46.217 INFO:teuthology.orchestra.run.smithi063.stdout:64424509442 2024-09-13T09:25:46.217 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:25:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:64424509443 2024-09-13T09:25:46.227 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:25:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:64424509445 2024-09-13T09:25:46.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:25:46.231 INFO:teuthology.orchestra.run.smithi063.stdout:64424509443 2024-09-13T09:25:46.232 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:25:46.233 INFO:teuthology.orchestra.run.smithi063.stdout:64424509444 2024-09-13T09:25:46.233 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:25:46.238 INFO:teuthology.orchestra.run.smithi063.stdout:60129542150 2024-09-13T09:25:46.238 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:25:46.671 INFO:teuthology.orchestra.run.smithi063.stdout:64424509441 2024-09-13T09:25:46.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509441 for osd.7 2024-09-13T09:25:46.694 INFO:teuthology.orchestra.run.smithi063.stdout:64424509442 2024-09-13T09:25:46.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.6 2024-09-13T09:25:46.717 INFO:teuthology.orchestra.run.smithi063.stdout:64424509443 2024-09-13T09:25:46.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.3 2024-09-13T09:25:46.734 INFO:teuthology.orchestra.run.smithi063.stdout:64424509441 2024-09-13T09:25:46.749 INFO:teuthology.orchestra.run.smithi063.stdout:64424509444 2024-09-13T09:25:46.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509441 for osd.0 2024-09-13T09:25:46.757 INFO:teuthology.orchestra.run.smithi063.stdout:64424509443 2024-09-13T09:25:46.758 INFO:teuthology.orchestra.run.smithi063.stdout:64424509442 2024-09-13T09:25:46.761 INFO:teuthology.orchestra.run.smithi063.stdout:60129542148 2024-09-13T09:25:46.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-13T09:25:46.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.1 2024-09-13T09:25:46.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.4 2024-09-13T09:25:46.774 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542148 for osd.2 2024-09-13T09:25:47.694 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:25:47.712 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:25:47.734 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:25:47.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:25:47.763 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:25:47.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:25:47.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:25:47.775 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:25:48.210 INFO:teuthology.orchestra.run.smithi063.stdout:64424509442 2024-09-13T09:25:48.223 INFO:teuthology.orchestra.run.smithi063.stdout:64424509444 2024-09-13T09:25:48.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509442 for osd.7 2024-09-13T09:25:48.230 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.231 INFO:teuthology.orchestra.run.smithi063.stdout:64424509443 2024-09-13T09:25:48.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.3 2024-09-13T09:25:48.242 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.6 2024-09-13T09:25:48.248 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.257 INFO:teuthology.orchestra.run.smithi063.stdout:64424509442 2024-09-13T09:25:48.262 INFO:teuthology.orchestra.run.smithi063.stdout:64424509445 2024-09-13T09:25:48.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509442 for osd.0 2024-09-13T09:25:48.270 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.270 INFO:teuthology.orchestra.run.smithi063.stdout:60129542150 2024-09-13T09:25:48.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-13T09:25:48.279 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-09-13T09:25:48.284 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.286 INFO:teuthology.orchestra.run.smithi063.stdout:64424509444 2024-09-13T09:25:48.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.4 2024-09-13T09:25:48.303 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.312 INFO:teuthology.orchestra.run.smithi063.stdout:64424509443 2024-09-13T09:25:48.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.1 2024-09-13T09:25:48.326 DEBUG:teuthology.parallel:result is None 2024-09-13T09:25:48.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:25:48.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:25:48.661 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:48.665 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:25:48.675 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-13T09:25:48.516468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221856,"kb_used_data":1568,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752864,"statfs":{"total":773094113280,"available":772866932736,"internally_reserved":0,"allocated":1605632,"data_stored":1648126,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000667"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109543+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:25:45.109543+0000","last_peered":"2024-09-13T09:25:45.109543+0000","last_clean":"2024-09-13T09:25:45.109543+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:25:45.109543+0000","last_undegraded":"2024-09-13T09:25:45.109543+0000","last_fullsized":"2024-09-13T09:25:45.109543+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109534+0000","last_change":"2024-09-13T09:25:45.109534+0000","last_active":"2024-09-13T09:25:45.109534+0000","last_peered":"2024-09-13T09:25:45.109534+0000","last_clean":"2024-09-13T09:25:45.109534+0000","last_became_active":"2024-09-13T09:25:43.394324+0000","last_became_peered":"2024-09-13T09:25:43.394324+0000","last_unstale":"2024-09-13T09:25:45.109534+0000","last_undegraded":"2024-09-13T09:25:45.109534+0000","last_fullsized":"2024-09-13T09:25:45.109534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:54.063959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109035+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:25:45.109035+0000","last_peered":"2024-09-13T09:25:45.109035+0000","last_clean":"2024-09-13T09:25:45.109035+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:25:45.109035+0000","last_undegraded":"2024-09-13T09:25:45.109035+0000","last_fullsized":"2024-09-13T09:25:45.109035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.961992+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:25:44.961992+0000","last_peered":"2024-09-13T09:25:44.961992+0000","last_clean":"2024-09-13T09:25:44.961992+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:25:44.961992+0000","last_undegraded":"2024-09-13T09:25:44.961992+0000","last_fullsized":"2024-09-13T09:25:44.961992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"19'117","reported_seq":160,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.802811+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:25:45.802811+0000","last_peered":"2024-09-13T09:25:45.802811+0000","last_clean":"2024-09-13T09:25:45.802811+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:25:45.802811+0000","last_undegraded":"2024-09-13T09:25:45.802811+0000","last_fullsized":"2024-09-13T09:25:45.802811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.975794+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:25:44.975794+0000","last_peered":"2024-09-13T09:25:44.975794+0000","last_clean":"2024-09-13T09:25:44.975794+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:25:44.975794+0000","last_undegraded":"2024-09-13T09:25:44.975794+0000","last_fullsized":"2024-09-13T09:25:44.975794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971366+0000","last_change":"2024-09-13T09:25:44.969750+0000","last_active":"2024-09-13T09:25:44.971366+0000","last_peered":"2024-09-13T09:25:44.971366+0000","last_clean":"2024-09-13T09:25:44.971366+0000","last_became_active":"2024-09-13T09:25:43.392036+0000","last_became_peered":"2024-09-13T09:25:43.392036+0000","last_unstale":"2024-09-13T09:25:44.971366+0000","last_undegraded":"2024-09-13T09:25:44.971366+0000","last_fullsized":"2024-09-13T09:25:44.971366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:30:55.268980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.006547+0000","last_change":"2024-09-13T09:25:45.006547+0000","last_active":"2024-09-13T09:25:45.006547+0000","last_peered":"2024-09-13T09:25:45.006547+0000","last_clean":"2024-09-13T09:25:45.006547+0000","last_became_active":"2024-09-13T09:25:43.392033+0000","last_became_peered":"2024-09-13T09:25:43.392033+0000","last_unstale":"2024-09-13T09:25:45.006547+0000","last_undegraded":"2024-09-13T09:25:45.006547+0000","last_fullsized":"2024-09-13T09:25:45.006547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:39.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971890+0000","last_change":"2024-09-13T09:25:44.971890+0000","last_active":"2024-09-13T09:25:44.971890+0000","last_peered":"2024-09-13T09:25:44.971890+0000","last_clean":"2024-09-13T09:25:44.971890+0000","last_became_active":"2024-09-13T09:25:43.394325+0000","last_became_peered":"2024-09-13T09:25:43.394325+0000","last_unstale":"2024-09-13T09:25:44.971890+0000","last_undegraded":"2024-09-13T09:25:44.971890+0000","last_fullsized":"2024-09-13T09:25:44.971890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:29.606233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":649040,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":649021,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":58672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:25:48.676 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:25:48.966 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:48.969 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:25:48.980 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-13T09:25:48.516468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221856,"kb_used_data":1568,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752864,"statfs":{"total":773094113280,"available":772866932736,"internally_reserved":0,"allocated":1605632,"data_stored":1648126,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000667"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109543+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:25:45.109543+0000","last_peered":"2024-09-13T09:25:45.109543+0000","last_clean":"2024-09-13T09:25:45.109543+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:25:45.109543+0000","last_undegraded":"2024-09-13T09:25:45.109543+0000","last_fullsized":"2024-09-13T09:25:45.109543+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109534+0000","last_change":"2024-09-13T09:25:45.109534+0000","last_active":"2024-09-13T09:25:45.109534+0000","last_peered":"2024-09-13T09:25:45.109534+0000","last_clean":"2024-09-13T09:25:45.109534+0000","last_became_active":"2024-09-13T09:25:43.394324+0000","last_became_peered":"2024-09-13T09:25:43.394324+0000","last_unstale":"2024-09-13T09:25:45.109534+0000","last_undegraded":"2024-09-13T09:25:45.109534+0000","last_fullsized":"2024-09-13T09:25:45.109534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:54.063959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109035+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:25:45.109035+0000","last_peered":"2024-09-13T09:25:45.109035+0000","last_clean":"2024-09-13T09:25:45.109035+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:25:45.109035+0000","last_undegraded":"2024-09-13T09:25:45.109035+0000","last_fullsized":"2024-09-13T09:25:45.109035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.961992+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:25:44.961992+0000","last_peered":"2024-09-13T09:25:44.961992+0000","last_clean":"2024-09-13T09:25:44.961992+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:25:44.961992+0000","last_undegraded":"2024-09-13T09:25:44.961992+0000","last_fullsized":"2024-09-13T09:25:44.961992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"19'117","reported_seq":160,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.802811+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:25:45.802811+0000","last_peered":"2024-09-13T09:25:45.802811+0000","last_clean":"2024-09-13T09:25:45.802811+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:25:45.802811+0000","last_undegraded":"2024-09-13T09:25:45.802811+0000","last_fullsized":"2024-09-13T09:25:45.802811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.975794+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:25:44.975794+0000","last_peered":"2024-09-13T09:25:44.975794+0000","last_clean":"2024-09-13T09:25:44.975794+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:25:44.975794+0000","last_undegraded":"2024-09-13T09:25:44.975794+0000","last_fullsized":"2024-09-13T09:25:44.975794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971366+0000","last_change":"2024-09-13T09:25:44.969750+0000","last_active":"2024-09-13T09:25:44.971366+0000","last_peered":"2024-09-13T09:25:44.971366+0000","last_clean":"2024-09-13T09:25:44.971366+0000","last_became_active":"2024-09-13T09:25:43.392036+0000","last_became_peered":"2024-09-13T09:25:43.392036+0000","last_unstale":"2024-09-13T09:25:44.971366+0000","last_undegraded":"2024-09-13T09:25:44.971366+0000","last_fullsized":"2024-09-13T09:25:44.971366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:30:55.268980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.006547+0000","last_change":"2024-09-13T09:25:45.006547+0000","last_active":"2024-09-13T09:25:45.006547+0000","last_peered":"2024-09-13T09:25:45.006547+0000","last_clean":"2024-09-13T09:25:45.006547+0000","last_became_active":"2024-09-13T09:25:43.392033+0000","last_became_peered":"2024-09-13T09:25:43.392033+0000","last_unstale":"2024-09-13T09:25:45.006547+0000","last_undegraded":"2024-09-13T09:25:45.006547+0000","last_fullsized":"2024-09-13T09:25:45.006547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:39.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971890+0000","last_change":"2024-09-13T09:25:44.971890+0000","last_active":"2024-09-13T09:25:44.971890+0000","last_peered":"2024-09-13T09:25:44.971890+0000","last_clean":"2024-09-13T09:25:44.971890+0000","last_became_active":"2024-09-13T09:25:43.394325+0000","last_became_peered":"2024-09-13T09:25:43.394325+0000","last_unstale":"2024-09-13T09:25:44.971890+0000","last_undegraded":"2024-09-13T09:25:44.971890+0000","last_fullsized":"2024-09-13T09:25:44.971890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:29.606233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":649040,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":649021,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":58672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:25:48.981 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:25:48.981 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T09:25:48.981 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T09:25:48.982 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T09:25:49.299 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:49.299 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T09:25:49.313 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T09:25:49.313 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T09:25:49.323 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T09:25:49.323 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd clone copy on read': True}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T09:25:49.323 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T09:25:49.324 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T09:25:49.648 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T09:25:49.648 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:25:49.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:25:49.946 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:49.949 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:25:49.961 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-13T09:25:48.516468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221856,"kb_used_data":1568,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752864,"statfs":{"total":773094113280,"available":772866932736,"internally_reserved":0,"allocated":1605632,"data_stored":1648126,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000667"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109543+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:25:45.109543+0000","last_peered":"2024-09-13T09:25:45.109543+0000","last_clean":"2024-09-13T09:25:45.109543+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:25:45.109543+0000","last_undegraded":"2024-09-13T09:25:45.109543+0000","last_fullsized":"2024-09-13T09:25:45.109543+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109534+0000","last_change":"2024-09-13T09:25:45.109534+0000","last_active":"2024-09-13T09:25:45.109534+0000","last_peered":"2024-09-13T09:25:45.109534+0000","last_clean":"2024-09-13T09:25:45.109534+0000","last_became_active":"2024-09-13T09:25:43.394324+0000","last_became_peered":"2024-09-13T09:25:43.394324+0000","last_unstale":"2024-09-13T09:25:45.109534+0000","last_undegraded":"2024-09-13T09:25:45.109534+0000","last_fullsized":"2024-09-13T09:25:45.109534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:54.063959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109035+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:25:45.109035+0000","last_peered":"2024-09-13T09:25:45.109035+0000","last_clean":"2024-09-13T09:25:45.109035+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:25:45.109035+0000","last_undegraded":"2024-09-13T09:25:45.109035+0000","last_fullsized":"2024-09-13T09:25:45.109035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.961992+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:25:44.961992+0000","last_peered":"2024-09-13T09:25:44.961992+0000","last_clean":"2024-09-13T09:25:44.961992+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:25:44.961992+0000","last_undegraded":"2024-09-13T09:25:44.961992+0000","last_fullsized":"2024-09-13T09:25:44.961992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"19'117","reported_seq":160,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.802811+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:25:45.802811+0000","last_peered":"2024-09-13T09:25:45.802811+0000","last_clean":"2024-09-13T09:25:45.802811+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:25:45.802811+0000","last_undegraded":"2024-09-13T09:25:45.802811+0000","last_fullsized":"2024-09-13T09:25:45.802811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.975794+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:25:44.975794+0000","last_peered":"2024-09-13T09:25:44.975794+0000","last_clean":"2024-09-13T09:25:44.975794+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:25:44.975794+0000","last_undegraded":"2024-09-13T09:25:44.975794+0000","last_fullsized":"2024-09-13T09:25:44.975794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971366+0000","last_change":"2024-09-13T09:25:44.969750+0000","last_active":"2024-09-13T09:25:44.971366+0000","last_peered":"2024-09-13T09:25:44.971366+0000","last_clean":"2024-09-13T09:25:44.971366+0000","last_became_active":"2024-09-13T09:25:43.392036+0000","last_became_peered":"2024-09-13T09:25:43.392036+0000","last_unstale":"2024-09-13T09:25:44.971366+0000","last_undegraded":"2024-09-13T09:25:44.971366+0000","last_fullsized":"2024-09-13T09:25:44.971366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:30:55.268980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.006547+0000","last_change":"2024-09-13T09:25:45.006547+0000","last_active":"2024-09-13T09:25:45.006547+0000","last_peered":"2024-09-13T09:25:45.006547+0000","last_clean":"2024-09-13T09:25:45.006547+0000","last_became_active":"2024-09-13T09:25:43.392033+0000","last_became_peered":"2024-09-13T09:25:43.392033+0000","last_unstale":"2024-09-13T09:25:45.006547+0000","last_undegraded":"2024-09-13T09:25:45.006547+0000","last_fullsized":"2024-09-13T09:25:45.006547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:39.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971890+0000","last_change":"2024-09-13T09:25:44.971890+0000","last_active":"2024-09-13T09:25:44.971890+0000","last_peered":"2024-09-13T09:25:44.971890+0000","last_clean":"2024-09-13T09:25:44.971890+0000","last_became_active":"2024-09-13T09:25:43.394325+0000","last_became_peered":"2024-09-13T09:25:43.394325+0000","last_unstale":"2024-09-13T09:25:44.971890+0000","last_undegraded":"2024-09-13T09:25:44.971890+0000","last_fullsized":"2024-09-13T09:25:44.971890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:29.606233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":649040,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":649021,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":58672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:25:49.962 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:25:50.265 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:50.268 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:25:50.281 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-13T09:25:50.135860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221856,"kb_used_data":1568,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752864,"statfs":{"total":773094113280,"available":772866932736,"internally_reserved":0,"allocated":1605632,"data_stored":1648126,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.620059"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109543+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:25:45.109543+0000","last_peered":"2024-09-13T09:25:45.109543+0000","last_clean":"2024-09-13T09:25:45.109543+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:25:45.109543+0000","last_undegraded":"2024-09-13T09:25:45.109543+0000","last_fullsized":"2024-09-13T09:25:45.109543+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109534+0000","last_change":"2024-09-13T09:25:45.109534+0000","last_active":"2024-09-13T09:25:45.109534+0000","last_peered":"2024-09-13T09:25:45.109534+0000","last_clean":"2024-09-13T09:25:45.109534+0000","last_became_active":"2024-09-13T09:25:43.394324+0000","last_became_peered":"2024-09-13T09:25:43.394324+0000","last_unstale":"2024-09-13T09:25:45.109534+0000","last_undegraded":"2024-09-13T09:25:45.109534+0000","last_fullsized":"2024-09-13T09:25:45.109534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:54.063959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109035+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:25:45.109035+0000","last_peered":"2024-09-13T09:25:45.109035+0000","last_clean":"2024-09-13T09:25:45.109035+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:25:45.109035+0000","last_undegraded":"2024-09-13T09:25:45.109035+0000","last_fullsized":"2024-09-13T09:25:45.109035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.961992+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:25:44.961992+0000","last_peered":"2024-09-13T09:25:44.961992+0000","last_clean":"2024-09-13T09:25:44.961992+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:25:44.961992+0000","last_undegraded":"2024-09-13T09:25:44.961992+0000","last_fullsized":"2024-09-13T09:25:44.961992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"19'117","reported_seq":160,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.802811+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:25:45.802811+0000","last_peered":"2024-09-13T09:25:45.802811+0000","last_clean":"2024-09-13T09:25:45.802811+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:25:45.802811+0000","last_undegraded":"2024-09-13T09:25:45.802811+0000","last_fullsized":"2024-09-13T09:25:45.802811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.975794+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:25:44.975794+0000","last_peered":"2024-09-13T09:25:44.975794+0000","last_clean":"2024-09-13T09:25:44.975794+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:25:44.975794+0000","last_undegraded":"2024-09-13T09:25:44.975794+0000","last_fullsized":"2024-09-13T09:25:44.975794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971366+0000","last_change":"2024-09-13T09:25:44.969750+0000","last_active":"2024-09-13T09:25:44.971366+0000","last_peered":"2024-09-13T09:25:44.971366+0000","last_clean":"2024-09-13T09:25:44.971366+0000","last_became_active":"2024-09-13T09:25:43.392036+0000","last_became_peered":"2024-09-13T09:25:43.392036+0000","last_unstale":"2024-09-13T09:25:44.971366+0000","last_undegraded":"2024-09-13T09:25:44.971366+0000","last_fullsized":"2024-09-13T09:25:44.971366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:30:55.268980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.006547+0000","last_change":"2024-09-13T09:25:45.006547+0000","last_active":"2024-09-13T09:25:45.006547+0000","last_peered":"2024-09-13T09:25:45.006547+0000","last_clean":"2024-09-13T09:25:45.006547+0000","last_became_active":"2024-09-13T09:25:43.392033+0000","last_became_peered":"2024-09-13T09:25:43.392033+0000","last_unstale":"2024-09-13T09:25:45.006547+0000","last_undegraded":"2024-09-13T09:25:45.006547+0000","last_fullsized":"2024-09-13T09:25:45.006547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:39.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971890+0000","last_change":"2024-09-13T09:25:44.971890+0000","last_active":"2024-09-13T09:25:44.971890+0000","last_peered":"2024-09-13T09:25:44.971890+0000","last_clean":"2024-09-13T09:25:44.971890+0000","last_became_active":"2024-09-13T09:25:43.394325+0000","last_became_peered":"2024-09-13T09:25:43.394325+0000","last_unstale":"2024-09-13T09:25:44.971890+0000","last_undegraded":"2024-09-13T09:25:44.971890+0000","last_fullsized":"2024-09-13T09:25:44.971890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:29.606233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":649040,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":649021,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":58672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:25:50.282 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:25:50.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 20 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:fsid eb7f03c3-9691-4766-8999-6f4f526cda71 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-09-13T09:25:30.379173+0000 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-09-13T09:25:50.125287+0000 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:crush_version 5 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:full_ratio 0.95 2024-09-13T09:25:50.595 INFO:teuthology.orchestra.run.smithi063.stdout:backfillfull_ratio 0.9 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:nearfull_ratio 0.85 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:require_min_compat_client luminous 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:min_compat_client jewel 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:require_osd_release reef 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:stretch_mode_enabled false 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:max_osd 8 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6816/2906107514,v1:172.21.15.63:6817/2906107514] [v2:172.21.15.63:6818/2906107514,v1:172.21.15.63:6819/2906107514] exists,up 54ea2bd8-55b0-4ded-8a16-041261c5ba5a 2024-09-13T09:25:50.596 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6824/3622854518,v1:172.21.15.63:6825/3622854518] [v2:172.21.15.63:6826/3622854518,v1:172.21.15.63:6827/3622854518] exists,up 0c95aee9-f871-4f6f-bbbd-1206db5d331c 2024-09-13T09:25:50.597 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6800/2105659441,v1:172.21.15.63:6801/2105659441] [v2:172.21.15.63:6802/2105659441,v1:172.21.15.63:6803/2105659441] exists,up f8253db2-d4da-46b5-95cf-cd5c9ebd7328 2024-09-13T09:25:50.597 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6808/4165134522,v1:172.21.15.63:6809/4165134522] [v2:172.21.15.63:6810/4165134522,v1:172.21.15.63:6811/4165134522] exists,up 67c11a52-baeb-4ea8-89ea-26018ffdc35a 2024-09-13T09:25:50.597 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/1789187713,v1:172.21.15.99:6801/1789187713] [v2:172.21.15.99:6802/1789187713,v1:172.21.15.99:6803/1789187713] exists,up 1cec9702-fa09-4d7c-b111-7d10b1950cfe 2024-09-13T09:25:50.597 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6809/3410135021,v1:172.21.15.99:6811/3410135021] [v2:172.21.15.99:6813/3410135021,v1:172.21.15.99:6815/3410135021] exists,up 7c0b908a-65c9-426c-867c-cc8ed65084b3 2024-09-13T09:25:50.597 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/4152410871,v1:172.21.15.99:6810/4152410871] [v2:172.21.15.99:6812/4152410871,v1:172.21.15.99:6814/4152410871] exists,up 78d5990a-4535-4110-ba04-88f01ab9fe28 2024-09-13T09:25:50.597 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/2923799929,v1:172.21.15.99:6825/2923799929] [v2:172.21.15.99:6826/2923799929,v1:172.21.15.99:6827/2923799929] exists,up 53f8511c-729f-4423-8eac-813c01f8f72b 2024-09-13T09:25:50.609 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6816/2906107514,v1:172.21.15.63:6817/2906107514] [v2:172.21.15.63:6818/2906107514,v1:172.21.15.63:6819/2906107514] exists,up 54ea2bd8-55b0-4ded-8a16-041261c5ba5a', 'osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6824/3622854518,v1:172.21.15.63:6825/3622854518] [v2:172.21.15.63:6826/3622854518,v1:172.21.15.63:6827/3622854518] exists,up 0c95aee9-f871-4f6f-bbbd-1206db5d331c', 'osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6800/2105659441,v1:172.21.15.63:6801/2105659441] [v2:172.21.15.63:6802/2105659441,v1:172.21.15.63:6803/2105659441] exists,up f8253db2-d4da-46b5-95cf-cd5c9ebd7328', 'osd.3 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.63:6808/4165134522,v1:172.21.15.63:6809/4165134522] [v2:172.21.15.63:6810/4165134522,v1:172.21.15.63:6811/4165134522] exists,up 67c11a52-baeb-4ea8-89ea-26018ffdc35a', 'osd.4 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/1789187713,v1:172.21.15.99:6801/1789187713] [v2:172.21.15.99:6802/1789187713,v1:172.21.15.99:6803/1789187713] exists,up 1cec9702-fa09-4d7c-b111-7d10b1950cfe', 'osd.5 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6809/3410135021,v1:172.21.15.99:6811/3410135021] [v2:172.21.15.99:6813/3410135021,v1:172.21.15.99:6815/3410135021] exists,up 7c0b908a-65c9-426c-867c-cc8ed65084b3', 'osd.6 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/4152410871,v1:172.21.15.99:6810/4152410871] [v2:172.21.15.99:6812/4152410871,v1:172.21.15.99:6814/4152410871] exists,up 78d5990a-4535-4110-ba04-88f01ab9fe28', 'osd.7 up in weight 1 up_from 15 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/2923799929,v1:172.21.15.99:6825/2923799929] [v2:172.21.15.99:6826/2923799929,v1:172.21.15.99:6827/2923799929] exists,up 53f8511c-729f-4423-8eac-813c01f8f72b'] 2024-09-13T09:25:50.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T09:25:50.715 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "mon.a", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster": "ceph", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "admin_socket_mode": "", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_client_required": "cephx, none", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_cluster_required": "cephx", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_debug": "true", 2024-09-13T09:25:50.716 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_service_required": "cephx", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "auth_supported": "cephx", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_aio": "true", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_aio_reap_max": "16", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_async_discard": "false", 2024-09-13T09:25:50.717 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_block_size": "4096", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_debug_aio": "false", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_enable_discard": "false", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_flock_retry": "3", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_inject_crash": "0", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_ioring": "false", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_ioring_hipri": "false", 2024-09-13T09:25:50.718 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bdev_type": "", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_allocator": "hybrid", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_buffered_io": "true", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T09:25:50.719 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_replay_recovery": "false", 2024-09-13T09:25:50.720 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluefs_sync_write": "false", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_allocator": "hybrid", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T09:25:50.721 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_create": "true", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_db_create": "false", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_db_path": "", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_db_size": "0", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_path": "", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_size": "107374182400", 2024-09-13T09:25:50.722 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_wal_create": "false", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_wal_path": "", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_bluefs": "true", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:25:50.723 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_autotune": "true", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_size": "0", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T09:25:50.724 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cache_type": "2q", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_clone_cow": "true", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T09:25:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_mode": "none", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_freelist": "false", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T09:25:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T09:25:50.727 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fail_eio": "false", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T09:25:50.728 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T09:25:50.729 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_max_blob_size": "0", 2024-09-13T09:25:50.730 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T09:25:50.731 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_spdk_mem": "512", 2024-09-13T09:25:50.732 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_tracing": "false", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T09:25:50.733 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T09:25:50.734 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T09:25:50.735 INFO:teuthology.orchestra.run.smithi063.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_cluster_require_version": "2", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_require_signatures": "false", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_require_version": "2", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_service_require_signatures": "false", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_service_require_version": "2", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "cephx_sign_messages": "true", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "chdir": "", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_acl_type": "", 2024-09-13T09:25:50.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_asio_thread_count": "2", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_cache_mid": "0.750000", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_cache_size": "16384", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_caps_release_delay": "5", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_check_pool_perm": "true", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_debug_force_sync_read": "false", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_debug_getattr_caps": "false", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_debug_inject_features": "", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_die_on_failed_remount": "false", 2024-09-13T09:25:50.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_dirsize_rbytes": "true", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_force_lazyio": "false", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_fs": "", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_inject_release_failure": "false", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_max_inline_size": "4096", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_mds_namespace": "", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_metadata": "", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_mount_gid": "-1", 2024-09-13T09:25:50.738 INFO:teuthology.orchestra.run.smithi063.stdout: "client_mount_timeout": "300", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_mount_uid": "-1", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_mountpoint": "/", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_notify_timeout": "10", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_oc": "true", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_oc_max_objects": "1000", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_oc_size": "209715200", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_permissions": "true", 2024-09-13T09:25:50.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_quota": "true", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_quota_df": "true", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_readahead_max_bytes": "0", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_readahead_max_periods": "4", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_readahead_min": "131072", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_reconnect_stale": "false", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_shutdown_timeout": "30", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_snapdir": ".snap", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_tick_interval": "1", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_trace": "", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T09:25:50.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_use_faked_inos": "false", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client_use_random_mds": "false", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_graylog": "false", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_graylog_port": "12201", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_monitors": "default=true", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_syslog": "false", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "clog_to_syslog_level": "info", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "-", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_network": "", 2024-09-13T09:25:50.741 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_network_interface": "", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "compressor_zlib_isal": "false", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "compressor_zlib_level": "5", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "compressor_zstd_level": "1", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T09:25:50.742 INFO:teuthology.orchestra.run.smithi063.stdout: "crimson_seastar_smp": "1", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_location": "", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_location_hook": "", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_location_hook_timeout": "10", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "daemonize": "false", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "daos_pool": "tank", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_alienstore": "0/5", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_asok": "1/5", 2024-09-13T09:25:50.743 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_asok_assert_abort": "false", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_auth": "20/20", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_bdev": "1/3", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_bluefs": "1/5", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_bluestore": "1/5", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_buffer": "0/1", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_cephsqlite": "0/5", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_client": "0/5", 2024-09-13T09:25:50.744 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_compressor": "1/5", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_context": "0/1", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_crush": "1/1", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_crypto": "1/5", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_cyanstore": "0/5", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_dpdk": "1/5", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_eventtrace": "1/5", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_filer": "0/1", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_filestore": "1/3", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_finisher": "1/1", 2024-09-13T09:25:50.745 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_fuse": "1/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_javaclient": "1/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_journal": "1/3", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_journaler": "0/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_kstore": "1/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_leveldb": "4/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_lockdep": "0/1", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mclock": "1/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mds": "1/5", 2024-09-13T09:25:50.746 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mds_balancer": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mds_locker": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mds_log": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mds_migrator": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_memstore": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mgr": "2/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mgrc": "1/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_mon": "20/20", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_monc": "0/10", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_ms": "1/1", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_none": "0/5", 2024-09-13T09:25:50.747 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_objclass": "0/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_objectcacher": "0/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_objecter": "0/1", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_optracker": "0/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_osd": "1/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_paxos": "20/20", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_perfcounter": "1/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_prioritycache": "1/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rados": "0/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rbd": "0/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T09:25:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rbd_replay": "0/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_refs": "0/0", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_reserver": "1/1", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rgw": "1/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rgw_access": "1/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rgw_flight": "1/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rgw_sync": "1/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_rocksdb": "4/5", 2024-09-13T09:25:50.749 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_backref": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_cache": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_device": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_epm": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_journal": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_lba": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_odata": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_omap": "0/5", 2024-09-13T09:25:50.750 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_onode": "0/5", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_t": "0/5", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_seastore_tm": "0/5", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_striper": "0/1", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_test": "0/5", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_throttle": "1/1", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_timer": "0/1", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "debug_tp": "0/5", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "device_failure_prediction_mode": "none", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T09:25:50.751 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "err_to_graylog": "false", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "err_to_journald": "false", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "err_to_stderr": "true", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "err_to_syslog": "false", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event_tracing": "false", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "exporter_http_port": "9926", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "exporter_prio_limit": "5", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "exporter_sort_metrics": "true", 2024-09-13T09:25:50.752 INFO:teuthology.orchestra.run.smithi063.stdout: "exporter_stats_period": "5", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "fake_statfs_for_testing": "0", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "fatal_signal_handlers": "true", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filer_max_purge_ops": "10", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filer_max_truncate_ops": "128", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_blackhole": "false", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_btrfs_snap": "true", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_caller_concurrency": "10", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T09:25:50.753 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_debug_omap_check": "false", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_debug_verify_split": "false", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_dump_file": "", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fadvise": "true", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fail_eio": "true", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fd_cache_size": "128", 2024-09-13T09:25:50.754 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fiemap": "false", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_inject_stall": "0", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_journal_parallel": "false", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_journal_trailing": "false", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_journal_writeahead": "false", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_kill_at": "0", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T09:25:50.755 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T09:25:50.756 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_merge_threshold": "-10", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_odsync_write": "false", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_omap_backend_path": "", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_op_threads": "2", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_punch_hole": "false", 2024-09-13T09:25:50.757 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_queue_max_ops": "50", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T09:25:50.758 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_seek_data_hole": "false", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_sloppy_crc": "false", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_splice": "false", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_split_multiple": "2", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_split_rand_factor": "20", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_update_to": "1000", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T09:25:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_xfs_extsize": "false", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "filestore_zfs_snap": "false", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "fio_dir": "/tmp/fio", 2024-09-13T09:25:50.760 INFO:teuthology.orchestra.run.smithi063.stdout: "fsid": "eb7f03c3-9691-4766-8999-6f4f526cda71", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_allow_other": "true", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_big_writes": "true", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_debug": "false", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_default_permissions": "false", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_disable_pagecache": "false", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_max_write": "0", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_multithreaded": "true", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_require_active_mds": "true", 2024-09-13T09:25:50.761 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_set_user_groups": "true", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_splice_move": "true", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_splice_read": "true", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_splice_write": "true", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "gss_target_name": "ceph", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_file": "", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_inject_failure": "0", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_interval": "5", 2024-09-13T09:25:50.762 INFO:teuthology.orchestra.run.smithi063.stdout: "host": "smithi063", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T09:25:50.763 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "inject_early_sigterm": "false", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "jaeger_agent_port": "6799", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "jaeger_tracing_enable": "false", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_aio": "true", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_align_min_size": "65536", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_block_align": "true", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_block_size": "4096", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_dio": "true", 2024-09-13T09:25:50.764 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_discard": "false", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_force_aio": "false", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_ignore_corruption": "false", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_max_write_entries": "100", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_replay_from": "0", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_write_header_frequency": "0", 2024-09-13T09:25:50.765 INFO:teuthology.orchestra.run.smithi063.stdout: "journal_zero_on_create": "false", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "journaler_prefetch_periods": "10", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "journaler_prezero_periods": "5", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "journaler_write_head_interval": "15", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "key": "", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "keyfile": "", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_backend": "rocksdb", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_max_bytes": "67108864", 2024-09-13T09:25:50.766 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_max_ops": "512", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_onode_map_size": "1024", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "kstore_sync_transaction": "false", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_block_size": "65536", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_bloom_size": "0", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_cache_size": "536870912", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_compression": "false", 2024-09-13T09:25:50.767 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_log": "", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_max_open_files": "0", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_paranoid": "false", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "librados_thread_count": "2", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "lockdep": "false", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "lockdep_force_backtrace": "false", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "log_coarse_timestamps": "true", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T09:25:50.768 INFO:teuthology.orchestra.run.smithi063.stdout: "log_flush_on_exit": "false", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_graylog_port": "12201", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_max_new": "1000", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_max_recent": "10000", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_stderr_prefix": "", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_file": "true", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_graylog": "false", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_journald": "false", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_stderr": "false", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "log_to_syslog": "false", 2024-09-13T09:25:50.769 INFO:teuthology.orchestra.run.smithi063.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_action_on_write_error": "1", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_alternate_name_max": "8192", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_asio_thread_count": "2", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_export_pin": "true", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T09:25:50.770 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_interval": "10", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_max": "-1", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_max_until": "-1", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_merge_size": "50", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_mode": "0", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T09:25:50.771 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_split_bits": "3", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_split_size": "10000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T09:25:50.772 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_mid": "0.700000", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_trim_interval": "1", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T09:25:50.773 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_debug_auth_pins": "false", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_debug_frag": "false", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_debug_scatterstat": "false", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_debug_subtrees": "false", 2024-09-13T09:25:50.774 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_default_dir_hash": "2", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_defer_session_stale": "true", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dir_max_entries": "0", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dir_prefetch": "true", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T09:25:50.775 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_early_reply": "true", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_enable_op_tracker": "true", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_enforce_unique_name": "true", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T09:25:50.776 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_inject_health_dummy": "false", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T09:25:50.777 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_join_fs": "", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_journal_format": "1", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_create_at": "0", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_export_at": "0", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_import_at": "0", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_journal_at": "0", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T09:25:50.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_link_at": "0", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_openc_at": "0", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_kill_rename_at": "0", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_max_events": "-1", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_max_segments": "128", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_pause": "false", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_segment_size": "0", 2024-09-13T09:25:50.779 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_completed_requests": "100000", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_export_size": "20971520", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_file_recover": "32", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_purge_files": "64", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_purge_ops": "8192", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T09:25:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_metrics_update_interval": "2", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_min_caps_per_client": "100", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_numa_node": "-1", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_op_history_duration": "600", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_op_history_size": "20", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T09:25:50.781 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_op_log_threshold": "5", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_ping_grace": "15", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_ping_interval": "5", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_recall_max_caps": "30000", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T09:25:50.782 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_replay_interval": "1.000000", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_root_ino_gid": "0", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_root_ino_uid": "0", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T09:25:50.783 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_shutdown_check": "0", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_skip_ino": "0", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_snap_min_uid": "0", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_snap_rstat": "false", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T09:25:50.784 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_symlink_recovery": "true", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_thrash_exports": "0", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_thrash_fragments": "0", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_tick_interval": "5.000000", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_valgrind_exit": "false", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_verify_backtrace": "1", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_verify_scatter": "false", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mds_wipe_sessions": "false", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mempool_debug": "false", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T09:25:50.785 INFO:teuthology.orchestra.run.smithi063.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "memstore_page_set": "false", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "memstore_page_size": "65536", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_client_bytes": "134217728", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_client_messages": "512", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T09:25:50.786 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_disabled_modules": "", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_mds_messages": "128", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_mon_messages": "128", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T09:25:50.787 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_osd_messages": "8192", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_pool": "true", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_standby_modules": "true", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_stats_period": "5", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_stats_threshold": "5", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_tick_period": "2", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_allow_pool_delete": "true", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T09:25:50.788 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_bytes": "104857600", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T09:25:50.789 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T09:25:50.790 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_compact_on_start": "false", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_compact_on_trim": "true", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T09:25:50.791 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_cpu_threads": "4", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_data_avail_crit": "5", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_data_avail_warn": "5", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_dump_json": "false", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T09:25:50.792 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_extra_checks": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_election_default_strategy": "1", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_election_timeout": "5.000000", 2024-09-13T09:25:50.793 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_enable_op_tracker": "true", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_fake_pool_delete": "false", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_force_quorum_join": "false", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_health_log_update_period": "5", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_health_max_detail": "50", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_health_to_clog": "true", 2024-09-13T09:25:50.794 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_host": "172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.99", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_host_override": "", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_initial_members": "", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_lease": "5.000000", 2024-09-13T09:25:50.795 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_log_full_interval": "50", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_log_max": "10000", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_log_max_summary": "50", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_log_epochs": "500", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_osd": "10000", 2024-09-13T09:25:50.796 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_memory_autotune": "true", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_memory_target": "2147483648", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mgr_digest_period": "5", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T09:25:50.797 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_op_complaint_time": "30", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_op_history_duration": "600", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_op_history_size": "20", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_op_log_threshold": "5", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T09:25:50.798 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_cache_size": "500", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T09:25:50.799 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T09:25:50.800 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_report_timeout": "900", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T09:25:50.801 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T09:25:50.802 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_reweight_max_osds": "4", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_scrub_interval": "86400", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_scrub_max_keys": "100", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_scrub_timeout": "300", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_session_timeout": "300", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_smart_report_timeout": "5", 2024-09-13T09:25:50.803 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_stretch_pool_size": "4", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_sync_debug": "false", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T09:25:50.804 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_tick_interval": "5", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T09:25:50.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_older_version": "true", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T09:25:50.806 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "monmap": "", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "motr_tracing_enabled": "false", 2024-09-13T09:25:50.807 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_op_threads": "3", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_cm": "false", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_device_name": "", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T09:25:50.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T09:25:50.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T09:25:50.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T09:25:50.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T09:25:50.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_sl": "3", 2024-09-13T09:25:50.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T09:25:50.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_rdma_type": "ib", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_async_reap_threshold": "5", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_before_connect": "false", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_ipv4": "true", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_ipv6": "false", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_msgr1": "true", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_msgr2": "true", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_port_max": "7568", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_port_min": "6800", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T09:25:50.813 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_retry_count": "3", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_bind_retry_delay": "5", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_blackhole_client": "false", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_blackhole_mds": "false", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_blackhole_mgr": "false", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_blackhole_mon": "false", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_blackhole_osd": "false", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_client_mode": "crc secure", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_cluster_type": "", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_compress_secure": "false", 2024-09-13T09:25:50.814 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_crc_data": "true", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_crc_header": "true", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_die_on_bug": "true", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_die_on_old_message": "true", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T09:25:50.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_hugepages": "", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_lro": "true", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T09:25:50.816 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_pmd": "", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_port_id": "0", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_dump_on_send": "false", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_inject_delay_type": "", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T09:25:50.817 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_inject_network_congestion": "0", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_max_accept_failures": "4", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_max_backoff": "15.000000", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_osd_compress_mode": "none", 2024-09-13T09:25:50.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_pq_min_cost": "65536", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_public_type": "", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_service_mode": "crc secure", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_tcp_nodelay": "true", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "ms_type": "async+posix", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "no_config_file": "false", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T09:25:50.819 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_inflight_ops": "1024", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objecter_timeout": "10.000000", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "objectstore_blackhole": "false", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "openssl_engine_opts": "", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T09:25:50.820 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_max_ops": "4", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_agent_slop": "0.020000", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_asio_thread_count": "2", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T09:25:50.821 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_backfill_scan_max": "512", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_backfill_scan_min": "64", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_backoff_on_peering": "false", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_beacon_report_interval": "300", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bench_duration": "30", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_blkin_trace_all": "false", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T09:25:50.822 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_class_update_on_start": "true", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_client_message_cap": "256", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T09:25:50.823 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_client_op_priority": "63", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_client_watch_timeout": "30", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_command_max_records": "256", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_compact_on_start": "false", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_crush_update_on_start": "true", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T09:25:50.824 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_op_order": "false", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T09:25:50.825 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_shutdown": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T09:25:50.826 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_default_notify_timeout": "30", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T09:25:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_enable_op_tracker": "true", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_fast_info": "true", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_fast_shutdown": "true", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T09:25:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_function_tracing": "false", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_grace": "20", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_interval": "6", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_stale": "600", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T09:25:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_journal_size": "5120", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_kill_backfill_at": "0", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_map_cache_size": "50", 2024-09-13T09:25:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_map_dedup": "true", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_map_message_max": "40", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_attr_name_len": "100", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_attr_size": "0", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_backfills": "1", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_markdown_count": "5", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_markdown_period": "600", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_object_name_len": "2048", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_object_size": "134217728", 2024-09-13T09:25:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_pgls": "1024", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_push_cost": "8388608", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_push_objects": "10", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_scrubs": "3", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T09:25:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_max_write_size": "90", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_profile": "balanced", 2024-09-13T09:25:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T09:25:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_base": "805306368", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_target": "4294967296", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_target_autotune": "false", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_min_recovery_priority": "0", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T09:25:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mon_report_interval": "5", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_num_cache_shards": "32", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_numa_node": "-1", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_objecter_finishers": "1", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_objectstore": "bluestore", 2024-09-13T09:25:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_objectstore_fuse": "false", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_objectstore_tracing": "false", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_history_duration": "600", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_history_size": "20", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_log_threshold": "5", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_num_shards": "0", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T09:25:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_op_thread_timeout": "15", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_open_classes_on_start": "true", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_os_flags": "0", 2024-09-13T09:25:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_peering_op_priority": "255", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_delete_priority": "5", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T09:25:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_crimson": "false", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T09:25:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_flags": "0", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_min_size": "0", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T09:25:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_size": "2", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_default_type": "replicated", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_cost": "20971520", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_max_active": "0", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T09:25:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_op_priority": "3", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_priority": "5", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T09:25:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_repair_during_recovery": "false", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T09:25:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_cost": "52428800", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_end_hour": "0", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T09:25:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_priority": "5", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T09:25:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_skip_data_digest": "false", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_smart_report_timeout": "5", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_snap_trim_priority": "5", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T09:25:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_target_transaction_size": "30", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T09:25:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_tracing": "false", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_use_stale_snap": "false", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_kill_at": "0", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_max_join_drift": "10", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_min": "500", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_min_wait": "0.050000", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_service_trim_max": "500", 2024-09-13T09:25:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_service_trim_min": "250", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_stash_full_interval": "25", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_trim_max": "500", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "paxos_trim_min": "250", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "perf": "true", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "-", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrv": "", 2024-09-13T09:25:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "public_bind_addr": "-", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "public_network": "", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "public_network_interface": "", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "qat_compressor_enabled": "false", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rados_mon_op_timeout": "0", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rados_osd_op_timeout": "0", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rados_tracing": "false", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_atime_update_interval": "60", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T09:25:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache": "true", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_policy": "writearound", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_size": "33554432", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T09:25:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_compression_hint": "none", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_clone_format": "auto", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_data_pool": "", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_features": "61", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_format": "2", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_map_options": "", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_order": "22", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_pool": "rbd", 2024-09-13T09:25:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_stripe_count": "0", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_io_scheduler": "simple", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T09:25:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_order": "24", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_pool": "", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_journal_splay_width": "4", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T09:25:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T09:25:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T09:25:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T09:25:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_op_threads": "1", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_plugins": "", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T09:25:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T09:25:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T09:25:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_tracing": "false", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_validate_names": "true", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd_validate_pool": "true", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "restapi_base_url": "", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "restapi_log_level": "", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_admin_entry": "admin", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_backend_store": "rados", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_barbican_url": "", 2024-09-13T09:25:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_beast_enable_async": "true", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_cache_enabled": "true", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T09:25:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_config_store": "rados", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_content_length_compat": "false", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_cross_domain_policy": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T09:25:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T09:25:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T09:25:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T09:25:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T09:25:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_log_window": "30", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T09:25:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T09:25:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dns_name": "", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dns_s3website_name": "", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_ops_log": "false", 2024-09-13T09:25:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_static_website": "false", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enable_usage_log": "false", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_expose_bucket": "false", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_extended_http_attrs": "", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_filter": "none", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T09:25:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_max_objs": "32", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T09:25:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_init_timeout": "300", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_password": "", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_project": "", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T09:25:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_token": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_admin_user": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_api_version": "2", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T09:25:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_url": "", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_max_objs": "32", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_max_worker": "3", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T09:25:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_log_http_headers": "", 2024-09-13T09:25:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_attr_size": "0", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T09:25:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_listing_results": "1000", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_notify_retries": "10", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T09:25:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T09:25:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_num_control_oids": "8", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_numa_node": "-1", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T09:25:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_op_tracing": "false", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_opa_token": "", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_opa_url": "", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ops_log_rados": "false", 2024-09-13T09:25:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_print_continue": "true", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T09:25:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_rados_tracing": "false", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_realm": "", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_realm_id": "", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_region": "", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T09:25:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_request_uri": "", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_resolve_cname": "false", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_run_sync_thread": "true", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T09:25:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_script_uri": "", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_service_provider_name": "", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_client_id": "", 2024-09-13T09:25:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_client_secret": "", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_entry": "sts", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_key": "sts", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_auth_url": "", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_custom_header": "", 2024-09-13T09:25:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_need_stats": "true", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_tenant_name": "", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_url": "", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_lease_period": "120", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T09:25:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_thread_pool_size": "512", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_comment": "", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_createby": "", 2024-09-13T09:25:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_encoding": "", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_flag": "false", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_origin": "", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_torrent_tracker": "", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_usage_max_shards": "32", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T09:25:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_use_opa_authz": "false", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_user_unique_email": "true", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_verify_ssl": "true", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T09:25:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_zone": "", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_zone_id": "", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_zonegroup": "", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_zonegroup_id": "", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_block_size": "4096", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T09:25:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T09:25:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_partition_filters": "false", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_perf": "false", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "run_dir": "/var/run/ceph", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_block_create": "true", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T09:25:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_device_size": "53687091200", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_main_device_type": "SSD", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T09:25:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "seastore_segment_size": "67108864", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "set_keepcaps": "false", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "setgroup": "", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "setuser": "", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "setuser_match_path": "", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "thp": "false", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "threadpool_default_timeout": "60", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "throttler_perf_counter": "true" 2024-09-13T09:25:50.896 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:50.899 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T09:25:51.045 INFO:teuthology.orchestra.run.smithi063.stdout:mon.a: {} 2024-09-13T09:25:51.061 INFO:teuthology.orchestra.run.smithi063.stdout:mon.b: {} 2024-09-13T09:25:51.074 INFO:teuthology.orchestra.run.smithi063.stdout:mon.c: {} 2024-09-13T09:25:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:25:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:25:51.077 INFO:teuthology.orchestra.run.smithi063.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:25:51.093 DEBUG:teuthology.orchestra.run.smithi063:> type ceph-objectstore-tool 2024-09-13T09:25:51.099 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:25:51.099 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-13T09:25:51.106 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:25:51.106 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T09:25:51.116 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T09:25:51.116 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T09:25:51.116 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:25:51.117 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:25:51.117 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:25:51.117 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T09:25:51.118 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T09:25:51.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:25:51.118 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T09:25:51.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:25:51.119 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T09:25:51.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:25:51.119 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T09:25:51.147 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:51.143+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:51.177 INFO:teuthology.orchestra.run.smithi063.stdout:mapped writes DISABLED 2024-09-13T09:25:51.177 INFO:teuthology.orchestra.run.smithi063.stdout:mapped reads DISABLED 2024-09-13T09:25:51.178 INFO:teuthology.orchestra.run.smithi063.stdout:Seed set to 108 2024-09-13T09:25:51.220 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:51.215+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:51.290 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:25:51.304 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:51.304 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:51.312 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:25:51.315 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:25:51.320 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:51.313+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:51.332 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:25:51.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:25:51.393 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:25:51.420 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:51.415+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:51.430 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:25:51.464 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:25:51.466 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:51.466 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:51.479 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:25:51.491 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:25:51.493 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:25:51.493 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:25:51.493 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:25:51.493 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:25:51.493 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:25:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:25:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:25:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:25:51.494 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:25:51.496 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:25:51.507 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-13T09:25:50.517085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221856,"kb_used_data":1568,"kb_used_omap":12,"kb_used_meta":220147,"kb_avail":754752864,"statfs":{"total":773094113280,"available":772866932736,"internally_reserved":0,"allocated":1605632,"data_stored":1648126,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12744,"internal_metadata":225431096},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001284"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109543+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:25:45.109543+0000","last_peered":"2024-09-13T09:25:45.109543+0000","last_clean":"2024-09-13T09:25:45.109543+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:25:45.109543+0000","last_undegraded":"2024-09-13T09:25:45.109543+0000","last_fullsized":"2024-09-13T09:25:45.109543+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109534+0000","last_change":"2024-09-13T09:25:45.109534+0000","last_active":"2024-09-13T09:25:45.109534+0000","last_peered":"2024-09-13T09:25:45.109534+0000","last_clean":"2024-09-13T09:25:45.109534+0000","last_became_active":"2024-09-13T09:25:43.394324+0000","last_became_peered":"2024-09-13T09:25:43.394324+0000","last_unstale":"2024-09-13T09:25:45.109534+0000","last_undegraded":"2024-09-13T09:25:45.109534+0000","last_fullsized":"2024-09-13T09:25:45.109534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:54.063959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.109035+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:25:45.109035+0000","last_peered":"2024-09-13T09:25:45.109035+0000","last_clean":"2024-09-13T09:25:45.109035+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:25:45.109035+0000","last_undegraded":"2024-09-13T09:25:45.109035+0000","last_fullsized":"2024-09-13T09:25:45.109035+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.961992+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:25:44.961992+0000","last_peered":"2024-09-13T09:25:44.961992+0000","last_clean":"2024-09-13T09:25:44.961992+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:25:44.961992+0000","last_undegraded":"2024-09-13T09:25:44.961992+0000","last_fullsized":"2024-09-13T09:25:44.961992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"19'117","reported_seq":160,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.802811+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:25:45.802811+0000","last_peered":"2024-09-13T09:25:45.802811+0000","last_clean":"2024-09-13T09:25:45.802811+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:25:45.802811+0000","last_undegraded":"2024-09-13T09:25:45.802811+0000","last_fullsized":"2024-09-13T09:25:45.802811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.975794+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:25:44.975794+0000","last_peered":"2024-09-13T09:25:44.975794+0000","last_clean":"2024-09-13T09:25:44.975794+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:25:44.975794+0000","last_undegraded":"2024-09-13T09:25:44.975794+0000","last_fullsized":"2024-09-13T09:25:44.975794+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"19'2","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971366+0000","last_change":"2024-09-13T09:25:44.969750+0000","last_active":"2024-09-13T09:25:44.971366+0000","last_peered":"2024-09-13T09:25:44.971366+0000","last_clean":"2024-09-13T09:25:44.971366+0000","last_became_active":"2024-09-13T09:25:43.392036+0000","last_became_peered":"2024-09-13T09:25:43.392036+0000","last_unstale":"2024-09-13T09:25:44.971366+0000","last_undegraded":"2024-09-13T09:25:44.971366+0000","last_fullsized":"2024-09-13T09:25:44.971366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:30:55.268980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:45.006547+0000","last_change":"2024-09-13T09:25:45.006547+0000","last_active":"2024-09-13T09:25:45.006547+0000","last_peered":"2024-09-13T09:25:45.006547+0000","last_clean":"2024-09-13T09:25:45.006547+0000","last_became_active":"2024-09-13T09:25:43.392033+0000","last_became_peered":"2024-09-13T09:25:43.392033+0000","last_unstale":"2024-09-13T09:25:45.006547+0000","last_undegraded":"2024-09-13T09:25:45.006547+0000","last_fullsized":"2024-09-13T09:25:45.006547+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:36:39.087597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-13T09:25:44.971890+0000","last_change":"2024-09-13T09:25:44.971890+0000","last_active":"2024-09-13T09:25:44.971890+0000","last_peered":"2024-09-13T09:25:44.971890+0000","last_clean":"2024-09-13T09:25:44.971890+0000","last_became_active":"2024-09-13T09:25:43.394325+0000","last_became_peered":"2024-09-13T09:25:43.394325+0000","last_unstale":"2024-09-13T09:25:44.971890+0000","last_undegraded":"2024-09-13T09:25:44.971890+0000","last_fullsized":"2024-09-13T09:25:44.971890+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:44:29.606233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":649040,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.139}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":57717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":649021,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":58672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31296,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340544,"statfs":{"total":96636764160,"available":96604717056,"internally_reserved":0,"allocated":184320,"data_stored":58653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.448}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2749999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:25:51.522 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:51.517+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:51.595 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:51.595 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:51.605 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:25:51.623 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:51.621+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:51.713 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:51.713 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:51.723 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:25:51.723 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:51.717+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:51.824 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:51.817+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:51.829 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:51.829 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:51.839 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:25:51.924 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:51.919+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:51.968 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:25:52.025 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:52.019+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:52.070 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.071 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.081 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:25:52.125 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:52.121+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:52.164 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:25:52.226 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:52.223+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:52.242 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.242 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.252 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:25:52.327 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:52.321+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:52.358 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.358 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.369 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:25:52.427 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:52.421+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:52.475 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.475 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.485 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:25:52.528 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:52.523+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:52.594 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.594 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.604 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:25:52.629 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:52.621+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:52.711 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.711 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:25:52.730 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:52.723+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:52.831 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:52.825+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:52.842 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.842 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:25:52.932 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:52.927+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:52.965 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:52.966 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:52.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:25:53.032 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:53.025+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:53.083 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:53.083 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:53.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:25:53.134 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:53.127+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:53.234 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:53.231+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:53.252 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:53.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:25:53.336 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:53.331+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:53.362 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:53.362 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:53.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:25:53.436 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:53.431+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:53.477 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:53.478 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:53.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:25:53.506 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:25:53.550 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:53.547+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:53.594 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:53.594 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:53.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:25:53.638 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:53.631+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:53.665 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:25:53.686 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:25:53.711 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:53.711 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:53.713 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:25:53.720 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:25:53.731 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:25:53.738 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:53.733+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:53.750 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:25:53.770 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:25:53.794 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:25:53.818 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:25:53.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:25:53.833 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:53.833 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:25:53.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:25:53.833 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:25:53.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.833 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4211.0:1 1.3 e17/16 1:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.955015+0000", 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.877478569000001, 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078310299999999995, 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955015+0000", 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955015+0000", 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955015+0000", 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955020+0000", 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:25:53.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955037+0000", 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6796999999999999e-05 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955046+0000", 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1849999999999993e-06 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955096+0000", 2024-09-13T09:25:53.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9564000000000002e-05 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955123+0000", 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7494000000000001e-05 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955758+0000", 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063526700000000001 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:53.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.955799+0000", 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0303999999999999e-05 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:53.838 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:53.840 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:53.833+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:53.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:25:53.951 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:53.933+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:54.004 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:54.004 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:25:54.004 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:25:54.004 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:54.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:25:54.043 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:54.039+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:54.127 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:54.127 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:25:54.128 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:25:54.128 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:25:54.128 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:54.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:25:54.142 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:54.135+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:54.176 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:25:54.243 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:54.239+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:54.251 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:25:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:25:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:25:54.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.2857222, 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.252 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:25:54.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.289332744999999, 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:25:54.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:25:54.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8812302259999996, 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0121125, 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 108 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365505+0000", 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365518+0000", 2024-09-13T09:25:54.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2456000000000001e-05 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365553+0000", 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5086000000000003e-05 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365567+0000", 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4187e-05 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371405+0000", 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058378960000000004 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371551+0000", 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014597300000000001 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371737+0000", 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018655899999999999 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377392+0000", 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056547380000000003 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377484+0000", 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2261000000000003e-05 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377565+0000", 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0464999999999996e-05 2024-09-13T09:25:54.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377619+0000", 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4221999999999997e-05 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8807591129999999, 2024-09-13T09:25:54.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014920113, 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365977+0000", 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365987+0000", 2024-09-13T09:25:54.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366004+0000", 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.662e-05 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366022+0000", 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7883000000000001e-05 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376091+0000", 2024-09-13T09:25:54.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010069171999999999 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376246+0000", 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000155377 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376548+0000", 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030137300000000002 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380798+0000", 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004250406 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380813+0000", 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4357e-05 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380862+0000", 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9137000000000001e-05 2024-09-13T09:25:54.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380898+0000", 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6158999999999997e-05 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8807005350000008, 2024-09-13T09:25:54.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016811104, 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366035+0000", 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366045+0000", 2024-09-13T09:25:54.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0327e-05 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366062+0000", 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6483000000000001e-05 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366074+0000", 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377874+0000", 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011799983999999999 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378006+0000", 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013130100000000001 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378143+0000", 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137157 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380914+0000", 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002771281 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382760+0000", 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001846253 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382810+0000", 2024-09-13T09:25:54.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9483e-05 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382847+0000", 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7749000000000001e-05 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.880620833, 2024-09-13T09:25:54.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018719919000000002, 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366115+0000", 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366313+0000", 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000198156 2024-09-13T09:25:54.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366331+0000", 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7739000000000001e-05 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366345+0000", 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4226000000000001e-05 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378684+0000", 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012339325 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378827+0000", 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014284700000000001 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.379116+0000", 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028930399999999999 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382675+0000", 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035590050000000001 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384568+0000", 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001892707 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384706+0000", 2024-09-13T09:25:54.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013804099999999999 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384836+0000", 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012977900000000001 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:25:54.287 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8803746750000006, 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01866932, 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:25:54.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366361+0000", 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366372+0000", 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0857e-05 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366388+0000", 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366401+0000", 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.326e-05 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380005+0000", 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013604036 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380100+0000", 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4468000000000004e-05 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380211+0000", 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011084199999999999 2024-09-13T09:25:54.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383662+0000", 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034510460000000001 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384905+0000", 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001243009 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384996+0000", 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385032+0000", 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5528000000000003e-05 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8803218259999994, 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020964449999999999, 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366414+0000", 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366423+0000", 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366443+0000", 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366455+0000", 2024-09-13T09:25:54.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380966+0000", 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014510662000000001 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381062+0000", 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6526000000000004e-05 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381259+0000", 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019658499999999999 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383675+0000", 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024160480000000001 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387295+0000", 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036198020000000001 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387342+0000", 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7299000000000002e-05 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387380+0000", 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7379000000000002e-05 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.299 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8798977990000001, 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020678247, 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366838+0000", 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367071+0000", 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023322800000000001 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367094+0000", 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3079999999999999e-05 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367111+0000", 2024-09-13T09:25:54.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7008999999999999e-05 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381788+0000", 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014676850999999999 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381894+0000", 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010610799999999999 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382045+0000", 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015073699999999999 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383689+0000", 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016436630000000001 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387445+0000", 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037564 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387488+0000", 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3402000000000002e-05 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387517+0000", 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8975999999999998e-05 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.305 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8796081119999997, 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020557408999999999, 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367128+0000", 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367138+0000", 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0672999999999999e-05 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367157+0000", 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8779000000000001e-05 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367174+0000", 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6464e-05 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382926+0000", 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015752759000000002 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383077+0000", 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150965 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383228+0000", 2024-09-13T09:25:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015101399999999999 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384458+0000", 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012294370000000001 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387599+0000", 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031411960000000002 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387647+0000", 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7648999999999997e-05 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387686+0000", 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9653000000000001e-05 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8795490069999996, 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022933500999999998, 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:25:54.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367187+0000", 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367208+0000", 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1793e-05 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367225+0000", 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668999999999999e-05 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367239+0000", 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4005999999999999e-05 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383748+0000", 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016508719000000002 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383933+0000", 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018481500000000001 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384122+0000", 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189626 2024-09-13T09:25:54.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385049+0000", 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092678599999999997 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390003+0000", 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049541819999999997 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390075+0000", 2024-09-13T09:25:54.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2094999999999993e-05 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390121+0000", 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6082999999999999e-05 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2896599989999995, 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012255703999999999, 2024-09-13T09:25:54.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 134 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957076+0000", 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957179+0000", 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103136 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957213+0000", 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4422999999999999e-05 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957225+0000", 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1178e-05 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963332+0000", 2024-09-13T09:25:54.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061077019999999996 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963543+0000", 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021082699999999999 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963933+0000", 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038984300000000001 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967237+0000", 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033042449999999999 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969266+0000", 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020291200000000001 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969303+0000", 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6183000000000002e-05 2024-09-13T09:25:54.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969333+0000", 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0063e-05 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2894936270000006, 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015204896000000001, 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 135 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957242+0000", 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957250+0000", 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957268+0000", 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7459999999999999e-05 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957277+0000", 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3559999999999999e-06 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965456+0000", 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792359999999995 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965567+0000", 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110877 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965741+0000", 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017349600000000001 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967304+0000", 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001563444 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972380+0000", 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050761019999999999 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972425+0000", 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4366000000000002e-05 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972448+0000", 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3455000000000002e-05 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2894495569999993, 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015208101, 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 136 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:25:54.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957297+0000", 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9909999999999997e-06 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957339+0000", 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2267e-05 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957371+0000", 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2518000000000003e-05 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966255+0000", 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088832679999999997 2024-09-13T09:25:54.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966353+0000", 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8658000000000003e-05 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966486+0000", 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013309500000000001 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969257+0000", 2024-09-13T09:25:54.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027707209999999999 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972453+0000", 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031962789999999998 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972478+0000", 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4655e-05 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972495+0000", 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7407000000000001e-05 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2893346959999992, 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015156871000000001, 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957401+0000", 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957509+0000", 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010808 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957536+0000", 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6237e-05 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957542+0000", 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2020000000000001e-06 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967352+0000", 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098099580000000006 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967447+0000", 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5375000000000001e-05 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967584+0000", 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136463 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971541+0000", 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039574930000000003 2024-09-13T09:25:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972499+0000", 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095777799999999995 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972527+0000", 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7900999999999999e-05 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972559+0000", 2024-09-13T09:25:54.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2191999999999999e-05 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2891848590000006, 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015104328, 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957551+0000", 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957556+0000", 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.318e-06 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957565+0000", 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.115e-06 2024-09-13T09:25:54.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957572+0000", 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968300+0000", 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010727617 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968387+0000", 2024-09-13T09:25:54.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7021000000000006e-05 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968482+0000", 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5081999999999995e-05 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971595+0000", 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031126639999999998 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972588+0000", 2024-09-13T09:25:54.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099296600000000007 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972635+0000", 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7136999999999999e-05 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972656+0000", 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1480000000000001e-05 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2891560579999997, 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015429492, 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-13T09:25:54.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957580+0000", 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957586+0000", 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957620+0000", 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3215e-05 2024-09-13T09:25:54.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957651+0000", 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1027e-05 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969570+0000", 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011919511000000001 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969674+0000", 2024-09-13T09:25:54.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104248 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969784+0000", 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010937700000000001 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972662+0000", 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028781589999999999 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972925+0000", 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026289899999999998 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972976+0000", 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0702999999999999e-05 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973010+0000", 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4743000000000003e-05 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2890514539999991, 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020098000000000001, 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-13T09:25:54.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957793+0000", 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108837 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957808+0000", 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-13T09:25:54.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957814+0000", 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970251+0000", 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012436523999999999 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970350+0000", 2024-09-13T09:25:54.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9600999999999997e-05 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970506+0000", 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015539500000000001 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972667+0000", 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002161619 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977681+0000", 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050135930000000002 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977722+0000", 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1634999999999997e-05 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977783+0000", 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0968000000000003e-05 2024-09-13T09:25:54.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2889153269999998, 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022700502000000001, 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957821+0000", 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957825+0000", 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957843+0000", 2024-09-13T09:25:54.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7492000000000001e-05 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957851+0000", 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971666+0000", 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013814707000000001 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971841+0000", 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174659 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971976+0000", 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013485600000000001 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973161+0000", 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011852130000000001 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980427+0000", 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072666440000000001 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980487+0000", 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9373000000000002e-05 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980522+0000", 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5244999999999999e-05 2024-09-13T09:25:54.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.4710473989999997, 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013540165999999999, 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:25:54.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775694+0000", 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8520000000000003e-06 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775699+0000", 2024-09-13T09:25:54.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.682e-06 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775709+0000", 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3980000000000005e-06 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783188+0000", 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074792560000000001 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783392+0000", 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020345199999999999 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783653+0000", 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026178699999999998 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.785173+0000", 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001519739 2024-09-13T09:25:54.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789105+0000", 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039318900000000004 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789174+0000", 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9568999999999997e-05 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789230+0000", 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5169e-05 2024-09-13T09:25:54.370 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:54.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:54.371 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:54.372 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:25:54.373 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:54.367+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:54.473 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:54.467+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:54.478 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:54.478 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:54.479 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:25:54.479 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:54.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:25:54.574 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:54.569+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:54.641 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:54.642 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:54.642 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:25:54.642 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:25:54.642 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:54.652 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:25:54.675 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:54.671+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.809731729999999, 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.763 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:54.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:25:54.767 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.7931735169999996, 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:54.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:54.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:54.771 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:54.772 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:54.772 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:54.772 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:54.772 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:54.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:25:54.776 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:54.769+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:54.876 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:54.871+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:54.880 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:54.880 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:54.880 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:25:54.880 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:54.889 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:25:54.977 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:54.969+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:55.001 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.001 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:55.001 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:25:55.001 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:25:55.001 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.011 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:25:55.078 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:55.073+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:55.123 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4211.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.168986287999999, 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051933830000000002, 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.124 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4211", 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 1 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:25:55.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948437+0000", 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948452+0000", 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4372999999999999e-05 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948471+0000", 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8932000000000001e-05 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948488+0000", 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7002000000000001e-05 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948542+0000", 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4128000000000001e-05 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948551+0000", 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1239999999999994e-06 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.951931+0000", 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003380245 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.952258+0000", 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032729100000000001 2024-09-13T09:25:55.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.952439+0000", 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018074399999999999 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953396+0000", 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00095699900000000004 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:55.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953544+0000", 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014766599999999999 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953591+0000", 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953634+0000", 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.130 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:54 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'32, mlcod=18'29)", 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.102775+0000", 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 11.014651024000001, 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036777749999999999, 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102775+0000", 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102775+0000", 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102774+0000", 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102789+0000", 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5588999999999999e-05 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102810+0000", 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0417000000000001e-05 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102833+0000", 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3615e-05 2024-09-13T09:25:55.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102915+0000", 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1595999999999996e-05 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.102997+0000", 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2400999999999999e-05 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.106386+0000", 2024-09-13T09:25:55.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033885550000000001 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.106453+0000", 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7205000000000005e-05 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:55 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'33, mlcod=18'29)", 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.103977+0000", 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 11.013449594000001, 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048140429999999996, 2024-09-13T09:25:55.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.103977+0000", 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.103977+0000", 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.103975+0000", 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104032+0000", 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6483000000000001e-05 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104063+0000", 2024-09-13T09:25:55.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0839000000000001e-05 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104075+0000", 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2303000000000001e-05 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104157+0000", 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1951000000000002e-05 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104199+0000", 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1766e-05 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.108684+0000", 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044854600000000001 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.108791+0000", 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010694800000000001 2024-09-13T09:25:55.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:56 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'34, mlcod=18'29)", 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 11.011880976, 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005743209, 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105544+0000", 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105557+0000", 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3464e-05 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105607+0000", 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9941999999999997e-05 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105620+0000", 2024-09-13T09:25:55.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3049e-05 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105726+0000", 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105185 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105793+0000", 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7164999999999998e-05 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.111223+0000", 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054298690000000004 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.111289+0000", 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6074999999999993e-05 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:69 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'43, mlcod=18'39)", 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.136389+0000", 2024-09-13T09:25:55.144 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.981037694999999, 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00352338, 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136389+0000", 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136389+0000", 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136387+0000", 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136443+0000", 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5930000000000002e-05 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136471+0000", 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7500000000000001e-05 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136483+0000", 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1871e-05 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136637+0000", 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000154029 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.136697+0000", 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0244999999999997e-05 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.139805+0000", 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003108576 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.139912+0000", 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010673799999999999 2024-09-13T09:25:55.148 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:85 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'55, mlcod=18'50)", 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.161262+0000", 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.956164657, 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003726704, 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161262+0000", 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161262+0000", 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161260+0000", 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161276+0000", 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.605e-05 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161314+0000", 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7858999999999998e-05 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161332+0000", 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7867e-05 2024-09-13T09:25:55.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161715+0000", 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000382894 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.161758+0000", 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2818999999999998e-05 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.164901+0000", 2024-09-13T09:25:55.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031428239999999998 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.164989+0000", 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7891000000000003e-05 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:112 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'74, mlcod=18'70)", 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.376909+0000", 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.740517177999999, 2024-09-13T09:25:55.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049794069999999999, 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376909+0000", 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376909+0000", 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376908+0000", 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376968+0000", 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0851999999999999e-05 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376998+0000", 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9346e-05 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377010+0000", 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2119999999999999e-05 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377111+0000", 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101004 2024-09-13T09:25:55.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377150+0000", 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9449999999999997e-05 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381831+0000", 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046802249999999997 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381889+0000", 2024-09-13T09:25:55.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8122999999999999e-05 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:113 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'75, mlcod=18'70)", 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.377936+0000", 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.739490147, 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040095970000000002, 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377936+0000", 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377936+0000", 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377934+0000", 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377953+0000", 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8519999999999999e-05 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377994+0000", 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1353000000000002e-05 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378019+0000", 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4919e-05 2024-09-13T09:25:55.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378106+0000", 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6744999999999997e-05 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378145+0000", 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9019e-05 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381892+0000", 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003747626 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381946+0000", 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3649000000000001e-05 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:114 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'76, mlcod=18'71)", 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:25:55.162 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.738461152999999, 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005632972, 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:25:55.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378964+0000", 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378976+0000", 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1652e-05 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378990+0000", 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.399e-05 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378999+0000", 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1349999999999998e-06 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379073+0000", 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4256000000000002e-05 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379142+0000", 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9139000000000003e-05 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384494+0000", 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053517829999999997 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384598+0000", 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010432 2024-09-13T09:25:55.166 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:115 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'77, mlcod=18'71)", 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.379781+0000", 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.737645177999999, 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048443180000000002, 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379781+0000", 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379781+0000", 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379780+0000", 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379858+0000", 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8008000000000003e-05 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379876+0000", 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7649999999999999e-05 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379884+0000", 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379954+0000", 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0223999999999997e-05 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379985+0000", 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1251000000000003e-05 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384602+0000", 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046171019999999997 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384626+0000", 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3428999999999999e-05 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.171 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:116 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'78, mlcod=18'72)", 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.380959+0000", 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.736467068, 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036880630000000001, 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.380959+0000", 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.380959+0000", 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.380958+0000", 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.380970+0000", 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1569999999999999e-05 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.380993+0000", 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3363999999999999e-05 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381003+0000", 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8099999999999992e-06 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381077+0000", 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3938999999999998e-05 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381105+0000", 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8614000000000001e-05 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384627+0000", 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035212759999999998 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384647+0000", 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0882000000000001e-05 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:117 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'79, mlcod=18'72)", 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.735574074000001, 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054346220000000001, 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381851+0000", 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381865+0000", 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4103e-05 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381883+0000", 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8522999999999999e-05 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381909+0000", 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5712999999999999e-05 2024-09-13T09:25:55.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381989+0000", 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0086000000000005e-05 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.382020+0000", 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0712e-05 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.387223+0000", 2024-09-13T09:25:55.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052033599999999998 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.387287+0000", 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3721999999999997e-05 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:119 2.0 e18/17 2:30306672:devicehealth::main.db.0000000000000000:head v 18'81, mlcod=18'78)", 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.389467+0000", 2024-09-13T09:25:55.180 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.727959637, 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035934059999999999, 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389467+0000", 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389467+0000", 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389465+0000", 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389601+0000", 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135857 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389623+0000", 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2272e-05 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389649+0000", 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5850000000000002e-05 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389726+0000", 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7293999999999996e-05 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.389757+0000", 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0590999999999999e-05 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.392947+0000", 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031899189999999998 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.393060+0000", 2024-09-13T09:25:55.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113346 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:140 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'98, mlcod=18'91)", 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.14925524, 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0068715920000000002, 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968170+0000", 2024-09-13T09:25:55.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968240+0000", 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9692999999999998e-05 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968254+0000", 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4268e-05 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968260+0000", 2024-09-13T09:25:55.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968307+0000", 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6836999999999999e-05 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968323+0000", 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6494000000000002e-05 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.974993+0000", 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066696510000000004 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.975043+0000", 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0002e-05 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:141 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'99, mlcod=18'91)", 2024-09-13T09:25:55.189 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.147707776000001, 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080457759999999993, 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969718+0000", 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969726+0000", 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969754+0000", 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8481000000000001e-05 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969764+0000", 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6509999999999996e-06 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969818+0000", 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4425000000000002e-05 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969848+0000", 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0122000000000001e-05 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.977718+0000", 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078694349999999993 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.977765+0000", 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6739000000000001e-05 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.193 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:143 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'101, mlcod=19'98)", 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.979595+0000", 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.137831744, 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004104931, 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979595+0000", 2024-09-13T09:25:55.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979595+0000", 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979594+0000", 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979684+0000", 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9913999999999993e-05 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979707+0000", 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3184999999999999e-05 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979717+0000", 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0360000000000001e-05 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979776+0000", 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8622000000000002e-05 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.979800+0000", 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4196e-05 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.983648+0000", 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038479600000000001 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.983700+0000", 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1731000000000003e-05 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:144 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'102, mlcod=19'98)", 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:25:55.198 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.136894735, 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058040629999999999, 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980531+0000", 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980635+0000", 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010367800000000001 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980690+0000", 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5795999999999999e-05 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980702+0000", 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1788e-05 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980759+0000", 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7141000000000001e-05 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980834+0000", 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4836e-05 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986281+0000", 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054468370000000004 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986336+0000", 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:25:55.202 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:145 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'103, mlcod=19'98)", 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.981315+0000", 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.136111578, 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050415989999999999, 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981315+0000", 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981315+0000", 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981314+0000", 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981396+0000", 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2196999999999999e-05 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981427+0000", 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1165999999999999e-05 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981445+0000", 2024-09-13T09:25:55.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8236999999999998e-05 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981494+0000", 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8520000000000003e-05 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981526+0000", 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1588999999999999e-05 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986340+0000", 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048145640000000003 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986357+0000", 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6359999999999999e-05 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:159 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'113, mlcod=19'107)", 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:25:55.207 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3361070420000001, 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051229429999999996, 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781327+0000", 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4819999999999994e-06 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781347+0000", 2024-09-13T09:25:55.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0279e-05 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781361+0000", 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3587999999999999e-05 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781410+0000", 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9589999999999998e-05 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781422+0000", 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1718e-05 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.786363+0000", 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049406010000000002 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.786442+0000", 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9585000000000005e-05 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:160 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'114, mlcod=19'112)", 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:45.788132+0000", 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3292943269999995, 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003854438, 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788132+0000", 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:55.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788132+0000", 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788120+0000", 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999876 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788170+0000", 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0040999999999997e-05 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788187+0000", 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7453e-05 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788194+0000", 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1760000000000004e-06 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788260+0000", 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6050000000000006e-05 2024-09-13T09:25:55.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.788284+0000", 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3879999999999998e-05 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.791930+0000", 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036461639999999999 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.791987+0000", 2024-09-13T09:25:55.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6301999999999997e-05 2024-09-13T09:25:55.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.216 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:55.216 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:55.216 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.217 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:25:55.218 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:25:55.219 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:55.211+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:55.318 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:55.311+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:55.374 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.374 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:55.375 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:25:55.375 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.385 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:25:55.430 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:55.427+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:55.488 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.488 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:55.488 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:25:55.488 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:25:55.488 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.499 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:25:55.520 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:55.517+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:55.606 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.606 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:25:55.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:25:55.606 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [] 2024-09-13T09:25:55.606 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:25:55.620 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:55.615+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:55.716 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:55.717 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:25:55.717 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.721 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:55.717+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:55.726 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:25:55.822 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:55.815+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:55.826 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:55.826 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:55.827 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:25:55.827 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:25:55.827 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:55.834 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:25:55.836 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:25:55.924 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:55.919+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:55.967 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:25:55.986 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:25:56.003 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:12 3.18 3:1a426f9e:::rbd_header.11182e510213:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.79848578999999997, 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0064571740000000004, 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:56.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 12 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204334+0000", 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204345+0000", 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0258000000000001e-05 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204358+0000", 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3867e-05 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204372+0000", 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3754000000000001e-05 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204436+0000", 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:25:56.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204440+0000", 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207020+0000", 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025802910000000002 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207135+0000", 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114299 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207374+0000", 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023949400000000001 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208859+0000", 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014852209999999999 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210662+0000", 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001802894 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210738+0000", 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5448e-05 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210793+0000", 2024-09-13T09:25:56.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5628999999999999e-05 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:12 3.18 e24/21 ondisk, result = 0)", 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.208765+0000", 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.79405703699999997, 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7220999999999996e-05, 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208765+0000", 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208765+0000", 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208764+0000", 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208773+0000", 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1290000000000002e-06 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208788+0000", 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5690000000000001e-05 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208801+0000", 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2660000000000001e-05 2024-09-13T09:25:56.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208851+0000", 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9781999999999999e-05 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208856+0000", 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2530000000000004e-06 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208862+0000", 2024-09-13T09:25:56.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9850000000000003e-06 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:16 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.225538+0000", 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77728390599999997, 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00040432899999999998, 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 16 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225538+0000", 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225538+0000", 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225536+0000", 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225548+0000", 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1837e-05 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225578+0000", 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9943000000000001e-05 2024-09-13T09:25:56.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225586+0000", 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1280000000000008e-06 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225630+0000", 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3954e-05 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225705+0000", 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4554999999999993e-05 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225942+0000", 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023761799999999999 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:17 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.226083+0000", 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77673866000000003, 2024-09-13T09:25:56.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026435400000000001, 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 17 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226083+0000", 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226083+0000", 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226082+0000", 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226089+0000", 2024-09-13T09:25:56.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9099999999999999e-06 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226105+0000", 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5447999999999998e-05 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226113+0000", 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.568e-06 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226164+0000", 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0711999999999998e-05 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226233+0000", 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9010000000000005e-05 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226348+0000", 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114781 2024-09-13T09:25:56.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:18 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.226472+0000", 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77635005199999996, 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030911099999999999, 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 18 2024-09-13T09:25:56.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226472+0000", 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226472+0000", 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226471+0000", 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226479+0000", 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3510000000000002e-06 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226495+0000", 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5738000000000001e-05 2024-09-13T09:25:56.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226503+0000", 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0609999999999996e-06 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226553+0000", 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0411000000000002e-05 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226619+0000", 2024-09-13T09:25:56.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6105000000000005e-05 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226781+0000", 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016154299999999999 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:19 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.226929+0000", 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77589292200000004, 2024-09-13T09:25:56.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00029894600000000002, 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 19 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226929+0000", 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226929+0000", 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226928+0000", 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226935+0000", 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6900000000000003e-06 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226950+0000", 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5458e-05 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226970+0000", 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0200999999999999e-05 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227024+0000", 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3393999999999997e-05 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227083+0000", 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8817000000000003e-05 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227228+0000", 2024-09-13T09:25:56.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014539300000000001 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:20 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.227876+0000", 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77494587100000001, 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071200899999999999, 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.036 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 20 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227876+0000", 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227876+0000", 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227875+0000", 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227884+0000", 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7310000000000005e-06 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227905+0000", 2024-09-13T09:25:56.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0364e-05 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227915+0000", 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8530000000000004e-06 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227990+0000", 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5706999999999999e-05 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228105+0000", 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114539 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228588+0000", 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00048332799999999999 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:21 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.228846+0000", 2024-09-13T09:25:56.040 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77397571899999995, 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00056186099999999998, 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 21 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228846+0000", 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228846+0000", 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228845+0000", 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228865+0000", 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0367e-05 2024-09-13T09:25:56.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228885+0000", 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9388000000000001e-05 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228894+0000", 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5799999999999998e-06 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228961+0000", 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7238999999999998e-05 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229057+0000", 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5124999999999995e-05 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229408+0000", 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00035147999999999999 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:22 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.229598+0000", 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77322409800000003, 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000416084, 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 22 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229598+0000", 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229598+0000", 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229596+0000", 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229606+0000", 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0799999999999995e-06 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229625+0000", 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8940000000000002e-05 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229639+0000", 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4435e-05 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229716+0000", 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7323000000000005e-05 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229810+0000", 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3943999999999994e-05 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.230014+0000", 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020373099999999999 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:24 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.231196+0000", 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77162608300000002, 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00040871100000000002, 2024-09-13T09:25:56.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 24 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231196+0000", 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231196+0000", 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231194+0000", 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231204+0000", 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8439999999999993e-06 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231224+0000", 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9746e-05 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231251+0000", 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6636999999999999e-05 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231314+0000", 2024-09-13T09:25:56.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3439e-05 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231405+0000", 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1100000000000005e-05 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231605+0000", 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000199316 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.053 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:25 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch watch cookie 140058279652592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.231936+0000", 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77088548499999998, 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019009179999999999, 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:56.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 25 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231936+0000", 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231936+0000", 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231935+0000", 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231944+0000", 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0040000000000005e-06 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231964+0000", 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9899999999999999e-05 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231974+0000", 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0108999999999999e-05 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232039+0000", 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4456999999999996e-05 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232136+0000", 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7668000000000001e-05 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232295+0000", 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000158326 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233436+0000", 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00114098 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233688+0000", 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025265299999999998 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233745+0000", 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6554000000000001e-05 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233837+0000", 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2670999999999994e-05 2024-09-13T09:25:56.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:25 3.18 e24/21 ondisk, result = 0)", 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.233578+0000", 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.76924436200000001, 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026949500000000002, 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233578+0000", 2024-09-13T09:25:56.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233578+0000", 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233576+0000", 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233587+0000", 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0872e-05 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233608+0000", 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0466e-05 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233618+0000", 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0631e-05 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233674+0000", 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5528000000000001e-05 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233683+0000", 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8210000000000008e-06 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233847+0000", 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000164549 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:56.064 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:56.064 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:56.065 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:25:56.066 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:25:56.066 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:25:56.066 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:25:56.066 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:56.057+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:56.087 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:25:56.110 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:25:56.112 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:25:56.112 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:25:56.112 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:25:56.112 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:25:56.112 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:25:56.112 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:25:56.113 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:25:56.113 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:25:56.166 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:56.161+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:56.179 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.179 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:25:56.267 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:56.263+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:56.290 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.290 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.300 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:25:56.368 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:56.361+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:56.401 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.401 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:25:56.469 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:56.465+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:56.508 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:25:56.509 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:25:56.509 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:25:56.509 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-13T09:25:56.562 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.562 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.570 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:56.565+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:56.571 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:25:56.671 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:56.667+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:56.724 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.724 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:25:56.772 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:56.765+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:56.818 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:25:56.811+0000 7f00df3a9700 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-09-13T09:25:56.842 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.842 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:25:56.873 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:56.869+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:56.961 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:56.961 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:56.971 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:25:57.014 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:57.009+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:57.075 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:57.069+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:57.122 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.123 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:25:57.176 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:57.171+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:57.201 INFO:teuthology.orchestra.run.smithi063.stderr:marked out osd.0. 2024-09-13T09:25:57.220 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:25:57.241 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.241 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:25:57.276 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:57.271+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:57.357 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.357 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:25:57.377 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:57.369+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:57.473 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.473 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.478 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:57.471+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:57.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:25:57.579 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:57.575+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:57.587 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.587 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.596 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:25:57.679 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:57.675+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:57.705 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.705 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:25:57.780 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:57.775+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:57.816 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.816 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.825 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:25:57.880 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:57.875+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:57.933 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:57.933 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:57.943 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:25:57.981 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:57.975+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:58.045 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:58.045 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:58.054 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:25:58.098 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:25:58.095+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:25:58.123 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:25:58.155 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:58.155 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:58.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:25:58.183 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:25:58.177+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:25:58.208 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:25:58.239 INFO:teuthology.orchestra.run.smithi063.stdout:skipping zero size read 2024-09-13T09:25:58.253 INFO:teuthology.orchestra.run.smithi063.stdout:2 write 0x3f2bcef thru 0x3f38ac6 (0xcdd8 bytes) 2024-09-13T09:25:58.260 INFO:teuthology.orchestra.run.smithi063.stdout:4 read 0x38f2e68 thru 0x38f7ba8 (0x4d41 bytes) 2024-09-13T09:25:58.260 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0x93ea2d7 2024-09-13T09:25:58.268 INFO:teuthology.orchestra.run.smithi063.stdout:5 trunc from 0x3f38ac7 to 0x93ea2d7 2024-09-13T09:25:58.277 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0xd269d1e 2024-09-13T09:25:58.282 INFO:teuthology.orchestra.run.smithi063.stdout:6 trunc from 0x93ea2d7 to 0xd269d1e 2024-09-13T09:25:58.283 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:25:58.284 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:58.279+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:58.292 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:58.292 INFO:teuthology.orchestra.run.smithi063.stdout:10 write 0xb30f44c thru 0xb31d637 (0xe1ec bytes) 2024-09-13T09:25:58.295 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:25:58.295 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:25:58.304 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:25:58.305 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:25:58.318 INFO:teuthology.orchestra.run.smithi063.stdout:12 trunc from 0xd269d1e to 0xcf45dd 2024-09-13T09:25:58.324 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:25:58.343 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:25:58.353 INFO:teuthology.orchestra.run.smithi063.stdout:13 write 0x4c24eb4 thru 0x4c27d54 (0x2ea1 bytes) 2024-09-13T09:25:58.359 INFO:teuthology.orchestra.run.smithi063.stdout:14 write 0x2920fcd thru 0x2925784 (0x47b8 bytes) 2024-09-13T09:25:58.366 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:25:58.384 INFO:teuthology.orchestra.run.smithi063.stdout:18 write 0x953f6da thru 0x9548d45 (0x966c bytes) 2024-09-13T09:25:58.384 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:58.377+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:58.389 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:25:58.402 INFO:teuthology.orchestra.run.smithi063.stdout:19 trunc from 0x9548d46 to 0x27e2487 2024-09-13T09:25:58.411 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:25:58.411 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:25:58.412 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:58.422 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:25:58.423 INFO:teuthology.orchestra.run.smithi063.stdout:20 clone 1 order 20 su 262144 sc 8 2024-09-13T09:25:58.433 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:25:58.435 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:25:58.436 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:25:58.491 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:58.487+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:58.523 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:58.524 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:58.524 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:25:58.524 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:25:58.524 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:58.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:25:58.587 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:25:58.581+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:25:58.637 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:58.637 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:25:58.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:25:58.637 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [] 2024-09-13T09:25:58.637 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:58.646 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:25:58.687 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:25:58.681+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:25:58.755 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:58.755 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:58.755 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:25:58.755 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:58.765 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:25:58.788 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:58.783+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:58.876 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:58.877 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:25:58.877 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:25:58.877 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:25:58.877 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:58.886 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:25:58.935 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:58.931+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:58.990 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:58.987+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:59.051 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:25:59.051 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:25:59.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:25:59.051 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:25:59.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:12 3.18 3:1a426f9e:::rbd_header.11182e510213:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8375558239999998, 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0064571740000000004, 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 12 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204334+0000", 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204345+0000", 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0258000000000001e-05 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204358+0000", 2024-09-13T09:25:59.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3867e-05 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204372+0000", 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3754000000000001e-05 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204436+0000", 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:25:59.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204440+0000", 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207020+0000", 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025802910000000002 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207135+0000", 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114299 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207374+0000", 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023949400000000001 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208859+0000", 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014852209999999999 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210662+0000", 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001802894 2024-09-13T09:25:59.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210738+0000", 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5448e-05 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210793+0000", 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5628999999999999e-05 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:16 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.225538+0000", 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8163539399999999, 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00040432899999999998, 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 16 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225538+0000", 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225538+0000", 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225536+0000", 2024-09-13T09:25:59.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225548+0000", 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1837e-05 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225578+0000", 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9943000000000001e-05 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225586+0000", 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1280000000000008e-06 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225630+0000", 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3954e-05 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225705+0000", 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4554999999999993e-05 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225942+0000", 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023761799999999999 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:18 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.226472+0000", 2024-09-13T09:25:59.063 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.815420086, 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030911099999999999, 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 18 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226472+0000", 2024-09-13T09:25:59.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226472+0000", 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226471+0000", 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226479+0000", 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3510000000000002e-06 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226495+0000", 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5738000000000001e-05 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226503+0000", 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0609999999999996e-06 2024-09-13T09:25:59.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226553+0000", 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0411000000000002e-05 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226619+0000", 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6105000000000005e-05 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.226781+0000", 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016154299999999999 2024-09-13T09:25:59.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:20 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.227876+0000", 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8140159049999998, 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071200899999999999, 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 20 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227876+0000", 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227876+0000", 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227875+0000", 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227884+0000", 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7310000000000005e-06 2024-09-13T09:25:59.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227905+0000", 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0364e-05 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227915+0000", 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8530000000000004e-06 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.227990+0000", 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5706999999999999e-05 2024-09-13T09:25:59.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228105+0000", 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114539 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228588+0000", 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00048332799999999999 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:21 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.228846+0000", 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8130457529999999, 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00056186099999999998, 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 21 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.073 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228846+0000", 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228846+0000", 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228845+0000", 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228865+0000", 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0367e-05 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228885+0000", 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9388000000000001e-05 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228894+0000", 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5799999999999998e-06 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.228961+0000", 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7238999999999998e-05 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229057+0000", 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5124999999999995e-05 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229408+0000", 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00035147999999999999 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.077 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:22 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.229598+0000", 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8122941319999999, 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000416084, 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 22 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229598+0000", 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229598+0000", 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229596+0000", 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229606+0000", 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0799999999999995e-06 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229625+0000", 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8940000000000002e-05 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229639+0000", 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4435e-05 2024-09-13T09:25:59.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229716+0000", 2024-09-13T09:25:59.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7323000000000005e-05 2024-09-13T09:25:59.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.229810+0000", 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3943999999999994e-05 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.230014+0000", 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020373099999999999 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.083 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:24 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.231196+0000", 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.810696117, 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00040871100000000002, 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 24 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231196+0000", 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231196+0000", 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231194+0000", 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231204+0000", 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8439999999999993e-06 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231224+0000", 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9746e-05 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231251+0000", 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6636999999999999e-05 2024-09-13T09:25:59.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231314+0000", 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3439e-05 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231405+0000", 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1100000000000005e-05 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231605+0000", 2024-09-13T09:25:59.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000199316 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:25 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch watch cookie 140058279652592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.231936+0000", 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8099555189999998, 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019009179999999999, 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 25 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231936+0000", 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231936+0000", 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231935+0000", 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231944+0000", 2024-09-13T09:25:59.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0040000000000005e-06 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231964+0000", 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9899999999999999e-05 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.231974+0000", 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0108999999999999e-05 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232039+0000", 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4456999999999996e-05 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232136+0000", 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7668000000000001e-05 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232295+0000", 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000158326 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233436+0000", 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00114098 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233688+0000", 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025265299999999998 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233745+0000", 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6554000000000001e-05 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233837+0000", 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2670999999999994e-05 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:106 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.094 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.344449+0000", 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.69744313199999997, 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020839869999999998, 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 106 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344449+0000", 2024-09-13T09:25:59.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344449+0000", 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344446+0000", 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344472+0000", 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6173000000000002e-05 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344504+0000", 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1816000000000002e-05 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344519+0000", 2024-09-13T09:25:59.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.507e-05 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344593+0000", 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3551000000000001e-05 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344719+0000", 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000126515 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.344977+0000", 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025793499999999999 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.345934+0000", 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00095696500000000005 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.346430+0000", 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00049630500000000005 2024-09-13T09:25:59.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.346487+0000", 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6351000000000003e-05 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.346533+0000", 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6037e-05 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:109 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.347449+0000", 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.694443013, 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00060605499999999998, 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 109 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.101 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347449+0000", 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347449+0000", 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347448+0000", 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347454+0000", 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1489999999999999e-06 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347467+0000", 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2347999999999999e-05 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347473+0000", 2024-09-13T09:25:59.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.764e-06 2024-09-13T09:25:59.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347533+0000", 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9778999999999998e-05 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.347631+0000", 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7447000000000004e-05 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348055+0000", 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00042447600000000002 2024-09-13T09:25:59.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:110 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.348251+0000", 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.69364133500000003, 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00035629200000000002, 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 110 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348251+0000", 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348251+0000", 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348250+0000", 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348256+0000", 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348273+0000", 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6487e-05 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348281+0000", 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9319999999999996e-06 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348326+0000", 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5930999999999998e-05 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348387+0000", 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0096999999999997e-05 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.348607+0000", 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022037700000000001 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:115 3.12 e26/25 3:49de9122:::rbd_data.11182e510213.0000000000000013:head v 26'1, mlcod=0'0)", 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.354735+0000", 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.68715693300000003, 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001402422, 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354735+0000", 2024-09-13T09:25:59.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354735+0000", 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354734+0000", 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354760+0000", 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5503999999999998e-05 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354774+0000", 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4358999999999999e-05 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354781+0000", 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0790000000000003e-06 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354836+0000", 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5155000000000003e-05 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.354863+0000", 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7339999999999999e-05 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.356103+0000", 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001239144 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.356137+0000", 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4878e-05 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:117 3.5 e26/21 3:a744e671:::rbd_data.11182e510213.000000000000000a:head v 26'3, mlcod=26'1)", 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.361806+0000", 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.68008609900000005, 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013814890000000001, 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361806+0000", 2024-09-13T09:25:59.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361806+0000", 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361805+0000", 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361829+0000", 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3988000000000001e-05 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361842+0000", 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3412e-05 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361855+0000", 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2771e-05 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361907+0000", 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2318000000000003e-05 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.361930+0000", 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2874000000000002e-05 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.363041+0000", 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001111508 2024-09-13T09:25:59.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.363187+0000", 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014589900000000001 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:119 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.376113+0000", 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.66577856400000002, 2024-09-13T09:25:59.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018990579999999999, 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 119 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376113+0000", 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376113+0000", 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376111+0000", 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376126+0000", 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5150999999999999e-05 2024-09-13T09:25:59.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376152+0000", 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5860999999999999e-05 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376166+0000", 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3559e-05 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376261+0000", 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5703000000000002e-05 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376406+0000", 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014450500000000001 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.376724+0000", 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00031781300000000001 2024-09-13T09:25:59.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.377775+0000", 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001051553 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.377957+0000", 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018149600000000001 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.377987+0000", 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0017000000000001e-05 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378012+0000", 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5656e-05 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:122 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.126 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.378792+0000", 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.663100099, 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00035541000000000001, 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 122 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378792+0000", 2024-09-13T09:25:59.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378792+0000", 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378791+0000", 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378797+0000", 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7019999999999998e-06 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378810+0000", 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3368e-05 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378817+0000", 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3540000000000002e-06 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378855+0000", 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7675000000000001e-05 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.378904+0000", 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389e-05 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379147+0000", 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024373799999999999 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.130 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.379301+0000", 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.66259099799999999, 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030275099999999997, 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123 2024-09-13T09:25:59.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379301+0000", 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379301+0000", 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379300+0000", 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379308+0000", 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8599999999999993e-06 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379321+0000", 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3287e-05 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379329+0000", 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6529999999999994e-06 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379367+0000", 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8448999999999997e-05 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379411+0000", 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3234e-05 2024-09-13T09:25:59.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.379604+0000", 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000193136 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:130 3.5 e26/21 3:a744e671:::rbd_data.11182e510213.000000000000000a:head v 26'4, mlcod=26'2)", 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.404325+0000", 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.63756715200000003, 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011495609999999999, 2024-09-13T09:25:59.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404325+0000", 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404325+0000", 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404324+0000", 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404336+0000", 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2407e-05 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404357+0000", 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0622999999999999e-05 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404366+0000", 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1770000000000004e-06 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404411+0000", 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5405999999999999e-05 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404433+0000", 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1990000000000001e-05 2024-09-13T09:25:59.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.405437+0000", 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001004186 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.405474+0000", 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6956999999999999e-05 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:131 3.12 e26/25 3:49de9122:::rbd_data.11182e510213.0000000000000013:head v 26'2, mlcod=0'0)", 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.404546+0000", 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.63734565399999998, 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033522259999999998, 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404546+0000", 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404546+0000", 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404546+0000", 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404554+0000", 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:25:59.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404579+0000", 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5778999999999998e-05 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404586+0000", 2024-09-13T09:25:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5910000000000001e-06 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404634+0000", 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7608999999999998e-05 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.404676+0000", 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2920999999999998e-05 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.407867+0000", 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031905760000000001 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.407899+0000", 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1541999999999997e-05 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:134 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.144 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.411896+0000", 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.62999641299999998, 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018484949999999999, 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 134 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.411896+0000", 2024-09-13T09:25:59.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.411896+0000", 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.411894+0000", 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.411904+0000", 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0225e-05 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.411926+0000", 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1851000000000001e-05 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.411939+0000", 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2869e-05 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.412006+0000", 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7533999999999994e-05 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.412108+0000", 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010132999999999999 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.412327+0000", 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000219483 2024-09-13T09:25:59.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.413233+0000", 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00090566300000000002 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.413646+0000", 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000413595 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.413698+0000", 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1817999999999998e-05 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.413744+0000", 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5757000000000003e-05 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:138 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:58.418309+0000", 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.62358334599999998, 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000371419, 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 138 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:25:59.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418309+0000", 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418309+0000", 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418308+0000", 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418314+0000", 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6259999999999997e-06 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418330+0000", 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5803000000000001e-05 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418339+0000", 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9970000000000004e-06 2024-09-13T09:25:59.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418377+0000", 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7446e-05 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418433+0000", 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6310000000000001e-05 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:58.418680+0000", 2024-09-13T09:25:59.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024715300000000001 2024-09-13T09:25:59.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.155 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:25:59.155 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:25:59.155 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:25:59.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:25:59.158 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:59.155+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:59.258 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:25:59.251+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:25:59.274 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:25:59.275 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:25:59.275 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:59.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:25:59.359 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:59.353+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:59.390 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:59.390 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:25:59.391 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:25:59.391 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:25:59.391 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:59.400 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:25:59.454 INFO:teuthology.orchestra.run.smithi063.stdout:22 write 0xecd2af3 thru 0xecd6079 (0x3587 bytes) 2024-09-13T09:25:59.460 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:59.455+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:59.486 INFO:teuthology.orchestra.run.smithi063.stdout:24 trunc from 0xecd607a to 0x266aafe 2024-09-13T09:25:59.509 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:59.509 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:54.169985+0000", 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.3374288100000005, 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040221709999999997, 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 7 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169985+0000", 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169985+0000", 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169983+0000", 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169996+0000", 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2428e-05 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170026+0000", 2024-09-13T09:25:59.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0026e-05 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170045+0000", 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9235000000000001e-05 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170160+0000", 2024-09-13T09:25:59.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011499 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170314+0000", 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015367199999999999 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170444+0000", 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013062999999999999 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:59.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.171457+0000", 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010129080000000001 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.173783+0000", 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023255279999999999 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.173905+0000", 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122707 2024-09-13T09:25:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.174007+0000", 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010195 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:55.191077+0000", 2024-09-13T09:25:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3163366180000002, 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040530949999999996, 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191077+0000", 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191077+0000", 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191075+0000", 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191088+0000", 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2452e-05 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191148+0000", 2024-09-13T09:25:59.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886999999999998e-05 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191171+0000", 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3524e-05 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191305+0000", 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133714 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191561+0000", 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025581399999999999 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191944+0000", 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038268500000000001 2024-09-13T09:25:59.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.194062+0000", 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021186960000000002 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.195002+0000", 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093997999999999998 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.195072+0000", 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9669e-05 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.195130+0000", 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:10 3.1c e24/21 3:3831950c:::rbd_directory:head v 24'1, mlcod=0'0)", 2024-09-13T09:25:59.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:55.200100+0000", 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3073136200000004, 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014617289999999999, 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200100+0000", 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200100+0000", 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200100+0000", 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200107+0000", 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8759999999999999e-06 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200140+0000", 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3225000000000001e-05 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200151+0000", 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0458e-05 2024-09-13T09:25:59.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200229+0000", 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8877999999999999e-05 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.200262+0000", 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2446000000000001e-05 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.201507+0000", 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012446149999999999 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.201562+0000", 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5611e-05 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:13 3.11 e24/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:25:59.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:55.219015+0000", 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2883991569999997, 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023358089999999999, 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219015+0000", 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219015+0000", 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219014+0000", 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219024+0000", 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5149999999999995e-06 2024-09-13T09:25:59.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219072+0000", 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8173000000000001e-05 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219088+0000", 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6507999999999998e-05 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219191+0000", 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010288100000000001 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.219241+0000", 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9379999999999998e-05 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.221277+0000", 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020363769999999998 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.221351+0000", 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3567999999999998e-05 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:118 3.11 e26/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 26'16, mlcod=26'14)", 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.376435+0000", 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1309794980000001, 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001617004, 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376435+0000", 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376435+0000", 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376433+0000", 2024-09-13T09:25:59.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376450+0000", 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7071999999999998e-05 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376484+0000", 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4603000000000001e-05 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376504+0000", 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9233e-05 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376633+0000", 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012898099999999999 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.376664+0000", 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1319000000000003e-05 2024-09-13T09:25:59.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.377930+0000", 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012664180000000001 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.378052+0000", 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121284 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:127 3.11 e26/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 26'17, mlcod=26'15)", 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.385409+0000", 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1220054820000001, 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001951061, 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385409+0000", 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385409+0000", 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385407+0000", 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385421+0000", 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3842999999999999e-05 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385443+0000", 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2257000000000001e-05 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385457+0000", 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3997e-05 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385566+0000", 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108309 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.385619+0000", 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2992999999999999e-05 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.387302+0000", 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001683527 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.387360+0000", 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7306000000000003e-05 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:128 3.6 e26/25 3:631b2480:::rbd_data.11182e510213.0000000000000025:head v 26'1, mlcod=0'0)", 2024-09-13T09:25:59.543 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.390101+0000", 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.117312568, 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021358589999999999, 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390101+0000", 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390101+0000", 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390101+0000", 2024-09-13T09:25:59.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390179+0000", 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7874999999999996e-05 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390205+0000", 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6186e-05 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390219+0000", 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390300+0000", 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1120999999999998e-05 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.390325+0000", 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5182999999999999e-05 2024-09-13T09:25:59.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.392121+0000", 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017954970000000001 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.392237+0000", 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116808 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:129 3.11 e26/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 26'18, mlcod=26'16)", 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.404023+0000", 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1033914119999999, 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001746355, 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404023+0000", 2024-09-13T09:25:59.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404023+0000", 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404020+0000", 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404040+0000", 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0216000000000001e-05 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404082+0000", 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1749000000000003e-05 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404097+0000", 2024-09-13T09:25:59.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4800000000000001e-05 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404203+0000", 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106266 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.404254+0000", 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0642000000000001e-05 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.405699+0000", 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014450019999999999 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.405769+0000", 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0006999999999996e-05 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:132 3.6 e26/25 3:631b2480:::rbd_data.11182e510213.0000000000000025:head v 26'2, mlcod=0'0)", 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.407069+0000", 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1003449569999999, 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018340699999999999, 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407069+0000", 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407069+0000", 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407067+0000", 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407086+0000", 2024-09-13T09:25:59.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9182e-05 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407124+0000", 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8033e-05 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407165+0000", 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1474999999999998e-05 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407284+0000", 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011818499999999999 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.407342+0000", 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8245000000000002e-05 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.408793+0000", 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014506899999999999 2024-09-13T09:25:59.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.408903+0000", 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011058899999999999 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:133 3.11 e26/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 26'19, mlcod=26'17)", 2024-09-13T09:25:59.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.412167+0000", 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.09524745, 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017810440000000001, 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412167+0000", 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412167+0000", 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412165+0000", 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412179+0000", 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.448e-05 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412215+0000", 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5975e-05 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412228+0000", 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3117e-05 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412351+0000", 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012243900000000001 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.412415+0000", 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3853e-05 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.413893+0000", 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014786210000000001 2024-09-13T09:25:59.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.413948+0000", 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4419000000000002e-05 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:135 3.11 e26/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 26'20, mlcod=26'18)", 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:58.417819+0000", 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0895946539999999, 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001705554, 2024-09-13T09:25:59.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417819+0000", 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417819+0000", 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417818+0000", 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417831+0000", 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2782000000000001e-05 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417853+0000", 2024-09-13T09:25:59.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2461999999999999e-05 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417863+0000", 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.417988+0000", 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012541200000000001 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.418050+0000", 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1499000000000002e-05 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.419448+0000", 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013983349999999999 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:58.419525+0000", 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6658999999999999e-05 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:148 3.f e27/21 3:f675f753:::rbd_object_map.11182e510213.0000000000000004:head v 27'1, mlcod=0'0)", 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.234599+0000", 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.27281523800000002, 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001889129, 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234599+0000", 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234599+0000", 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234598+0000", 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234607+0000", 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2430000000000005e-06 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234626+0000", 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9462000000000001e-05 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234636+0000", 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0281e-05 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234732+0000", 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5509999999999999e-05 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.234800+0000", 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7679e-05 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.236424+0000", 2024-09-13T09:25:59.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016244320000000001 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.236488+0000", 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3918999999999996e-05 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:180 3.1c e27/21 3:3831950c:::rbd_directory:head v 27'2, mlcod=0'0)", 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.277235+0000", 2024-09-13T09:25:59.572 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.23017884699999999, 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011580049999999999, 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277235+0000", 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277235+0000", 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277235+0000", 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277245+0000", 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0305999999999999e-05 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277299+0000", 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4119000000000001e-05 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277315+0000", 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5801999999999999e-05 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277387+0000", 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1960999999999998e-05 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.277413+0000", 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6084999999999998e-05 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.278343+0000", 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093030000000000001 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.278393+0000", 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9932000000000002e-05 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:184 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'3, mlcod=24'1)", 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.289408+0000", 2024-09-13T09:25:59.577 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.21800582700000001, 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018928149999999999, 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289408+0000", 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289408+0000", 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289406+0000", 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289422+0000", 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5454000000000001e-05 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289458+0000", 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5850999999999999e-05 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289477+0000", 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9211e-05 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289610+0000", 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013351500000000001 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.289680+0000", 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9536000000000006e-05 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.291208+0000", 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015281260000000001 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.291301+0000", 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2864999999999999e-05 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:199 3.a e27/21 3:535385f0:::rbd_children:head v 27'1, mlcod=0'0)", 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.318113+0000", 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.18930112599999999, 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001499422, 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318113+0000", 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318113+0000", 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318112+0000", 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318152+0000", 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0046000000000002e-05 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318217+0000", 2024-09-13T09:25:59.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5072000000000003e-05 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318242+0000", 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5185999999999999e-05 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318350+0000", 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107768 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.318389+0000", 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8621999999999997e-05 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319549+0000", 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011606139999999999 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319612+0000", 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3051000000000003e-05 2024-09-13T09:25:59.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:214 3.11 e27/21 3:89d17d3b:::rbd_object_map.11182e510213:head v 27'22, mlcod=26'20)", 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.370397+0000", 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.13701685, 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001469447, 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370397+0000", 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370397+0000", 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370395+0000", 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370411+0000", 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5512e-05 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370445+0000", 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3909999999999999e-05 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370459+0000", 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4742999999999999e-05 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370592+0000", 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000132547 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.370654+0000", 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2055000000000001e-05 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.371783+0000", 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011288750000000001 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.371867+0000", 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3684999999999996e-05 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:246 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'4, mlcod=27'2)", 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.437007+0000", 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.070406784, 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012227239999999999, 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437007+0000", 2024-09-13T09:25:59.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437007+0000", 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437006+0000", 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437019+0000", 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3611999999999999e-05 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437047+0000", 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7512e-05 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437057+0000", 2024-09-13T09:25:59.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0665999999999999e-05 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437142+0000", 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4141999999999995e-05 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.437177+0000", 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5475999999999997e-05 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.438162+0000", 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00098526699999999996 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.438230+0000", 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7682999999999996e-05 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.596 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:249 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'5, mlcod=27'3)", 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.440549+0000", 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.066864949000000007, 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004070756, 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440549+0000", 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440549+0000", 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440548+0000", 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440561+0000", 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3890000000000001e-05 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440586+0000", 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4286000000000001e-05 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440598+0000", 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2276000000000001e-05 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440702+0000", 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103905 2024-09-13T09:25:59.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440774+0000", 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1871000000000004e-05 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444541+0000", 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037672560000000001 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444620+0000", 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8794999999999994e-05 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:259 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'6, mlcod=27'4)", 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.454870+0000", 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.052544251, 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019910829999999998, 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.601 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.454870+0000", 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.454870+0000", 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.454868+0000", 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.454884+0000", 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5583999999999999e-05 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.454912+0000", 2024-09-13T09:25:59.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8731999999999999e-05 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.454924+0000", 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1917e-05 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.455043+0000", 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118967 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.455111+0000", 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7688000000000007e-05 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.456766+0000", 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001655557 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.456861+0000", 2024-09-13T09:25:59.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4364999999999995e-05 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:260 3.a e27/21 3:55f6faff:::rbd_data.111867511a81.00000000000000eb:head v 27'2, mlcod=0'0)", 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.458456+0000", 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.048957541, 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016375269999999999, 2024-09-13T09:25:59.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458456+0000", 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458456+0000", 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458454+0000", 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458517+0000", 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3206000000000004e-05 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458551+0000", 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4038999999999997e-05 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458569+0000", 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7425999999999999e-05 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458679+0000", 2024-09-13T09:25:59.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110517 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458725+0000", 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5645000000000001e-05 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460038+0000", 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013129319999999999 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460094+0000", 2024-09-13T09:25:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6141999999999999e-05 2024-09-13T09:25:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.612 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.612 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:59.614 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0xe9bfb60 2024-09-13T09:25:59.614 INFO:teuthology.orchestra.run.smithi063.stdout:25 trunc from 0x266aafe to 0xe9bfb60 2024-09-13T09:25:59.614 INFO:teuthology.orchestra.run.smithi063.stdout:26 writesame 0x5eefd18 thru 0x5effbec data_size 0x1(0xfed5 bytes) 2024-09-13T09:25:59.614 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:59.614 INFO:teuthology.orchestra.run.smithi063.stdout:28 write 0x709c904 thru 0x70a98b3 (0xcfb0 bytes) 2024-09-13T09:25:59.614 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:59.607+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:59.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:25:59.615 INFO:teuthology.orchestra.run.smithi063.stdout:29 write 0xc8a0645 thru 0xc8b014f (0xfb0b bytes) 2024-09-13T09:25:59.619 INFO:teuthology.orchestra.run.smithi063.stdout:30 writesame 0x3cf61fc thru 0x3d0378e data_size 0x1b(0xd593 bytes) 2024-09-13T09:25:59.628 INFO:teuthology.orchestra.run.smithi063.stdout:31 read 0x2c0ccd2 thru 0x2c18766 (0xba95 bytes) 2024-09-13T09:25:59.628 INFO:teuthology.orchestra.run.smithi063.stdout:32 write 0x81f0fc6 thru 0x81f1699 (0x6d4 bytes) 2024-09-13T09:25:59.635 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:59.635 INFO:teuthology.orchestra.run.smithi063.stdout:36 write 0xadfe756 thru 0xae00184 (0x1a2f bytes) 2024-09-13T09:25:59.641 INFO:teuthology.orchestra.run.smithi063.stdout:37 write 0x7899c73 thru 0x78a2a56 (0x8de4 bytes) 2024-09-13T09:25:59.648 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:59.648 INFO:teuthology.orchestra.run.smithi063.stdout:39 write 0xe133a2a thru 0xe14382f (0xfe06 bytes) 2024-09-13T09:25:59.655 INFO:teuthology.orchestra.run.smithi063.stdout:41 read 0xab503da thru 0xab5a2c4 (0x9eeb bytes) 2024-09-13T09:25:59.673 INFO:teuthology.orchestra.run.smithi063.stdout:42 trunc from 0xe9bfb60 to 0x6ec1bec 2024-09-13T09:25:59.708 INFO:teuthology.orchestra.run.smithi063.stdout:43 trunc from 0x6ec1bec to 0x17db04c 2024-09-13T09:25:59.714 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:25:59.709+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:25:59.735 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:59.735 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:25:59.735 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:25:59.735 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:59.736 INFO:teuthology.orchestra.run.smithi063.stdout:44 read 0xe3a68 thru 0xec7ed (0x8d86 bytes) 2024-09-13T09:25:59.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:25:59.757 INFO:teuthology.orchestra.run.smithi063.stdout:46 write 0x5996670 thru 0x5999f2f (0x38c0 bytes) 2024-09-13T09:25:59.783 INFO:teuthology.orchestra.run.smithi063.stdout:47 write 0x69ab183 thru 0x69b0919 (0x5797 bytes) 2024-09-13T09:25:59.801 INFO:teuthology.orchestra.run.smithi063.stdout:48 trunc from 0x69b091a to 0xb2edced 2024-09-13T09:25:59.814 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:59.815 INFO:teuthology.orchestra.run.smithi063.stdout:50 read 0xa47e8cf thru 0xa489943 (0xb075 bytes) 2024-09-13T09:25:59.815 INFO:teuthology.orchestra.run.smithi063.stdout:52 write 0x6d53308 thru 0x6d57e2f (0x4b28 bytes) 2024-09-13T09:25:59.815 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:25:59.811+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:25:59.834 INFO:teuthology.orchestra.run.smithi063.stdout:53 trunc from 0xb2edced to 0x57867d7 2024-09-13T09:25:59.853 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:59.853 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:25:59.853 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:25:59.853 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:25:59.853 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:25:59.855 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:59.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:25:59.874 INFO:teuthology.orchestra.run.smithi063.stdout:56 write 0x89f3fcc thru 0x89f4e90 (0xec5 bytes) 2024-09-13T09:25:59.881 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:25:59.881 INFO:teuthology.orchestra.run.smithi063.stdout:58 writesame 0x7cdd6f8 thru 0x7cea47b data_size 0xe(0xcd84 bytes) 2024-09-13T09:25:59.889 INFO:teuthology.orchestra.run.smithi063.stdout:59 read 0x1c68d5d thru 0x1c6bbd0 (0x2e74 bytes) 2024-09-13T09:25:59.889 INFO:teuthology.orchestra.run.smithi063.stdout:60 write 0x7c2ab7a thru 0x7c34736 (0x9bbd bytes) 2024-09-13T09:25:59.901 INFO:teuthology.orchestra.run.smithi063.stdout:62 trunc from 0x89f4e91 to 0x61ffac0 2024-09-13T09:25:59.915 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:25:59.911+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:25:59.941 INFO:teuthology.orchestra.run.smithi063.stdout:63 trunc from 0x61ffac0 to 0xe68decc 2024-09-13T09:25:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:64 writesame 0x85b009 thru 0x868d40 data_size 0x2(0xdd38 bytes) 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:25:59.968 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:55.196322+0000", 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7672786770000002, 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070426509999999996, 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 10 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196322+0000", 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196322+0000", 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196320+0000", 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196329+0000", 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2110000000000004e-06 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196342+0000", 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2714000000000001e-05 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196358+0000", 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5600999999999999e-05 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196466+0000", 2024-09-13T09:25:59.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010874699999999999 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.196474+0000", 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2819999999999999e-06 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.199345+0000", 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028711589999999999 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.199523+0000", 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017873100000000001 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.199849+0000", 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032587700000000002 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.201722+0000", 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001872325 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015242459999999999 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.203311+0000", 2024-09-13T09:25:59.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4777999999999995e-05 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.203364+0000", 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3758999999999998e-05 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:249 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.439931+0000", 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.52366935599999997, 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050683899999999999, 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 249 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.439931+0000", 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.439931+0000", 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.439930+0000", 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.439939+0000", 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4669999999999993e-06 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.439959+0000", 2024-09-13T09:25:59.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219000000000001e-05 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.439985+0000", 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.603e-05 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440044+0000", 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8356000000000002e-05 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440137+0000", 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3187000000000001e-05 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440440+0000", 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030292699999999998 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.441694+0000", 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012536839999999999 2024-09-13T09:25:59.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444827+0000", 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031333310000000001 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444905+0000", 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7595000000000005e-05 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444999+0000", 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4957999999999994e-05 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:261 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.486450+0000", 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.47715080599999998, 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004000165, 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 261 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486450+0000", 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486450+0000", 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486447+0000", 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486466+0000", 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9358999999999999e-05 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486504+0000", 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7713000000000003e-05 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486522+0000", 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7819e-05 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486678+0000", 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015595 2024-09-13T09:25:59.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.486953+0000", 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027577100000000001 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.488082+0000", 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001128448 2024-09-13T09:25:59.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490083+0000", 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002000702 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490316+0000", 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023287500000000001 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490394+0000", 2024-09-13T09:25:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8220999999999995e-05 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490450+0000", 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6063000000000002e-05 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:265 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497378+0000", 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46622251599999998, 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064833790000000001, 2024-09-13T09:25:59.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 265 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497378+0000", 2024-09-13T09:25:59.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497378+0000", 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497377+0000", 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497390+0000", 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3084e-05 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497401+0000", 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0890000000000001e-05 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497409+0000", 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9629999999999998e-06 2024-09-13T09:25:59.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498234+0000", 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00082503700000000004 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498309+0000", 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4919999999999994e-05 2024-09-13T09:25:59.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498799+0000", 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048997600000000004 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503786+0000", 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049873929999999997 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503797+0000", 2024-09-13T09:25:59.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1062e-05 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503830+0000", 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2564999999999997e-05 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503861+0000", 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1463e-05 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:25:59.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:266 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497421+0000", 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46617902700000002, 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066499330000000002, 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:25:59.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 266 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497421+0000", 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497421+0000", 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497421+0000", 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497426+0000", 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1499999999999998e-06 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497439+0000", 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3784000000000001e-05 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497463+0000", 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3652e-05 2024-09-13T09:25:59.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499415+0000", 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019517650000000001 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499482+0000", 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6623000000000002e-05 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499750+0000", 2024-09-13T09:25:59.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026862400000000002 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503899+0000", 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041486079999999998 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503966+0000", 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7008999999999995e-05 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504007+0000", 2024-09-13T09:26:00.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0915999999999999e-05 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504071+0000", 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4655999999999999e-05 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:267 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497475+0000", 2024-09-13T09:26:00.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46612514199999999, 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067003499999999999, 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 267 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497475+0000", 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497475+0000", 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497475+0000", 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497480+0000", 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6400000000000002e-06 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497491+0000", 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0413e-05 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497497+0000", 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4699999999999999e-06 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500302+0000", 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002804431 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500373+0000", 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1644999999999996e-05 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500653+0000", 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027975899999999998 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503952+0000", 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032991560000000001 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504091+0000", 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000138584 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504142+0000", 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0958999999999998e-05 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504176+0000", 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3976000000000001e-05 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:268 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.007 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497543+0000", 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46605734300000001, 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067330749999999998, 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 268 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497543+0000", 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497543+0000", 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497542+0000", 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497565+0000", 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2459e-05 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497576+0000", 2024-09-13T09:26:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1087000000000001e-05 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497587+0000", 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1316e-05 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.501088+0000", 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035007189999999998 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.501156+0000", 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7755000000000004e-05 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.501471+0000", 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000315746 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504192+0000", 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027205879999999999 2024-09-13T09:26:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504226+0000", 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3793e-05 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504253+0000", 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7401000000000001e-05 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504276+0000", 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3061000000000002e-05 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:269 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497615+0000", 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46598566499999999, 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077520569999999997, 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 269 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497615+0000", 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497615+0000", 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497614+0000", 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497623+0000", 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2499999999999995e-06 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497645+0000", 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1614e-05 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497659+0000", 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4054e-05 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.502763+0000", 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051036010000000001 2024-09-13T09:26:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.502842+0000", 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9503000000000001e-05 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.503200+0000", 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000357692 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505308+0000", 2024-09-13T09:26:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021080040000000001 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505312+0000", 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3089999999999997e-06 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505338+0000", 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5998000000000001e-05 2024-09-13T09:26:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505367+0000", 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8782e-05 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:270 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46590358399999998, 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087584770000000006, 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 270 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497696+0000", 2024-09-13T09:26:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497707+0000", 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1214e-05 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497716+0000", 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0250000000000008e-06 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497723+0000", 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504669+0000", 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069460399999999997 2024-09-13T09:26:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504713+0000", 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3582999999999999e-05 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504887+0000", 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174196 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505909+0000", 2024-09-13T09:26:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001022565 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506362+0000", 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045264300000000002 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506415+0000", 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2852000000000002e-05 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506455+0000", 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0556000000000003e-05 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:271 3.c 3:341b3506:::rbd_data.111867511a81.0000000000000021:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 961278] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.500573+0000", 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.46302782199999998, 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059389619999999999, 2024-09-13T09:26:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 271 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500573+0000", 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500573+0000", 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500571+0000", 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500582+0000", 2024-09-13T09:26:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0944999999999999e-05 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500602+0000", 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0205000000000001e-05 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500613+0000", 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1406000000000001e-05 2024-09-13T09:26:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504309+0000", 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036953789999999999 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504401+0000", 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2486999999999997e-05 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504500+0000", 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9204999999999996e-05 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505377+0000", 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087710900000000005 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506429+0000", 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001051031 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506473+0000", 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4716999999999999e-05 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506512+0000", 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8328999999999998e-05 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:287 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.543103+0000", 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42049699000000001, 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036495379999999999, 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 287 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543103+0000", 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543103+0000", 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543101+0000", 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543114+0000", 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.254e-05 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543145+0000", 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0828999999999999e-05 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543161+0000", 2024-09-13T09:26:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543288+0000", 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012693700000000001 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.543543+0000", 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025482100000000002 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.544299+0000", 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00075612699999999999 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.546055+0000", 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017560169999999999 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.546597+0000", 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054242500000000003 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.546680+0000", 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2930999999999996e-05 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.546753+0000", 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2835000000000003e-05 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:313 3.5 3:a5299b1c:::rbd_data.111867511a81.000000000000007a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 105587~36324] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.644825+0000", 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.3187758, 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033539770000000002, 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 313 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.644825+0000", 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.644825+0000", 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.644823+0000", 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.644941+0000", 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000117377 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.644959+0000", 2024-09-13T09:26:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8802000000000001e-05 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.644969+0000", 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.329e-06 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.645021+0000", 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2549000000000001e-05 2024-09-13T09:26:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.645170+0000", 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000148632 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.645334+0000", 2024-09-13T09:26:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000164284 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.646538+0000", 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001203939 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648083+0000", 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001544875 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648141+0000", 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8477000000000003e-05 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648179+0000", 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7218999999999997e-05 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:315 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.648805+0000", 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.31479573100000002, 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034249390000000001, 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 315 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648805+0000", 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648805+0000", 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648804+0000", 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648808+0000", 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9700000000000001e-06 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648820+0000", 2024-09-13T09:26:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1866e-05 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.648825+0000", 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0359999999999997e-06 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.649821+0000", 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099576799999999996 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.650050+0000", 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000229347 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.650676+0000", 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00062544399999999996 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.651985+0000", 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013097600000000001 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.652106+0000", 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120315 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.652160+0000", 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4051000000000001e-05 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.652230+0000", 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9895999999999996e-05 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:320 3.5 3:a5299b1c:::rbd_data.111867511a81.000000000000007a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.677012+0000", 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.28658812700000003, 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049852459999999996, 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 320 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677012+0000", 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677012+0000", 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677011+0000", 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677021+0000", 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0628000000000001e-05 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677045+0000", 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3847999999999999e-05 2024-09-13T09:26:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677059+0000", 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3526000000000001e-05 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677169+0000", 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110282 2024-09-13T09:26:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677407+0000", 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023838 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677710+0000", 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030268200000000003 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678843+0000", 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001133504 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.681885+0000", 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030415580000000002 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.681949+0000", 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4416000000000001e-05 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.681998+0000", 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8145000000000001e-05 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:338 3.c 3:341b3506:::rbd_data.111867511a81.0000000000000021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712281+0000", 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.25131921299999999, 2024-09-13T09:26:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0052165989999999997, 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 338 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712281+0000", 2024-09-13T09:26:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712281+0000", 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712279+0000", 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712291+0000", 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.19e-05 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712322+0000", 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1047000000000003e-05 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712345+0000", 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2416999999999999e-05 2024-09-13T09:26:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712450+0000", 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105764 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712553+0000", 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010217000000000001 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712791+0000", 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023832399999999999 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716070+0000", 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032786629999999998 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717384+0000", 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001314714 2024-09-13T09:26:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717448+0000", 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3905000000000006e-05 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717498+0000", 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9620000000000003e-05 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:347 3.10 e27/25 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head v 27'2, mlcod=0'0)", 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712885+0000", 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.25071547999999999, 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037220019999999999, 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712885+0000", 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712885+0000", 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712884+0000", 2024-09-13T09:26:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712893+0000", 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4849999999999992e-06 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712902+0000", 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7820000000000005e-06 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712909+0000", 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6950000000000003e-06 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713006+0000", 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7054999999999998e-05 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713053+0000", 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7239e-05 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716580+0000", 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035267150000000001 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716607+0000", 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6831000000000001e-05 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:342 3.2 e27/21 3:46f3aa3c:::rbd_data.111867511a81.0000000000000025:head v 27'3, mlcod=27'1)", 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.713225+0000", 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.25037574699999998, 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041880750000000003, 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713225+0000", 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713225+0000", 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713224+0000", 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713234+0000", 2024-09-13T09:26:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0862e-05 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713250+0000", 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5724000000000001e-05 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713459+0000", 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000208939 2024-09-13T09:26:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713530+0000", 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1229000000000005e-05 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713550+0000", 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9615000000000001e-05 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717380+0000", 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038304670000000002 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717413+0000", 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2357000000000002e-05 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:371 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.757414+0000", 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.20618656899999999, 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054934220000000004, 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 371 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757414+0000", 2024-09-13T09:26:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757414+0000", 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757412+0000", 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757426+0000", 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4e-05 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757464+0000", 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8380000000000002e-05 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757482+0000", 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7787000000000001e-05 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757614+0000", 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013198600000000001 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.757765+0000", 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015169099999999999 2024-09-13T09:26:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.758384+0000", 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061824099999999995 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.760096+0000", 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001712547 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.762761+0000", 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026651880000000002 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.762851+0000", 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9572000000000002e-05 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.762907+0000", 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6335999999999997e-05 2024-09-13T09:26:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:373 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.769544+0000", 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.19405673900000001, 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042953059999999996, 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 373 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769544+0000", 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769544+0000", 2024-09-13T09:26:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769542+0000", 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769552+0000", 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5230000000000001e-06 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769576+0000", 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.429e-05 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769590+0000", 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4304000000000001e-05 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769644+0000", 2024-09-13T09:26:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3573999999999999e-05 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.769755+0000", 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111548 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.770100+0000", 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034450300000000002 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.771469+0000", 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001368721 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.773604+0000", 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021356489999999999 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.773744+0000", 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013944599999999999 2024-09-13T09:26:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.773839+0000", 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5198000000000006e-05 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:395 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.834703+0000", 2024-09-13T09:26:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.12889737600000001, 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035961859999999999, 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 395 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834703+0000", 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834703+0000", 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834701+0000", 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834718+0000", 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7419999999999999e-05 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834756+0000", 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7385000000000002e-05 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834772+0000", 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5917e-05 2024-09-13T09:26:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.834903+0000", 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000131354 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.835195+0000", 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029170299999999998 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.836313+0000", 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001118225 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.837919+0000", 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001606374 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838142+0000", 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022303199999999999 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838230+0000", 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8151000000000004e-05 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838299+0000", 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8894999999999997e-05 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.098 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:00.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:00.100 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:00.093+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:00.101 INFO:teuthology.orchestra.run.smithi063.stdout:65 read 0x21b7b1f thru 0x21bcee1 (0x53c3 bytes) 2024-09-13T09:26:00.101 INFO:teuthology.orchestra.run.smithi063.stdout:66 punch from 0x68e6aab to 0x68ea736, (0x3c8b bytes) 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:69 trunc from 0xe68decc to 0x61621dd 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:70 write 0x5795c9c thru 0x57a4007 (0xe36c bytes) 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:71 writesame 0x4921061 thru 0x492ce2f data_size 0x3(0xbdcf bytes) 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:72 read 0x5ffdfd4 thru 0x6008e0a (0xae37 bytes) 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:74 read 0x2145006 thru 0x214f333 (0xa32e bytes) 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:76 write 0x63daec3 thru 0x63e78c1 (0xc9ff bytes) 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:77 trunc from 0x63e78c2 to 0x7ed1384 2024-09-13T09:26:00.102 INFO:teuthology.orchestra.run.smithi063.stdout:78 trunc from 0x7ed1384 to 0x8de8f9e 2024-09-13T09:26:00.103 INFO:teuthology.orchestra.run.smithi063.stdout:79 punch from 0x4d327a3 to 0x4d32b1f, (0x37c bytes) 2024-09-13T09:26:00.103 INFO:teuthology.orchestra.run.smithi063.stdout:80 write 0x95bea68 thru 0x95c5047 (0x65e0 bytes) 2024-09-13T09:26:00.103 INFO:teuthology.orchestra.run.smithi063.stdout:81 write 0x5775fe4 thru 0x5778768 (0x2785 bytes) 2024-09-13T09:26:00.106 INFO:teuthology.orchestra.run.smithi063.stdout:82 write 0x79a80e3 thru 0x79b5a50 (0xd96e bytes) 2024-09-13T09:26:00.112 INFO:teuthology.orchestra.run.smithi063.stdout:85 read 0x6a4bf0d thru 0x6a5304d (0x7141 bytes) 2024-09-13T09:26:00.113 INFO:teuthology.orchestra.run.smithi063.stdout:86 punch from 0x76e20b9 to 0x76e3d05, (0x1c4c bytes) 2024-09-13T09:26:00.113 INFO:teuthology.orchestra.run.smithi063.stdout:87 read 0x2616428 thru 0x2622b25 (0xc6fe bytes) 2024-09-13T09:26:00.113 INFO:teuthology.orchestra.run.smithi063.stdout:88 punch from 0x243f060 to 0x2445397, (0x6337 bytes) 2024-09-13T09:26:00.131 INFO:teuthology.orchestra.run.smithi063.stdout:89 write 0xd2121f4 thru 0xd21cba5 (0xa9b2 bytes) 2024-09-13T09:26:00.141 INFO:teuthology.orchestra.run.smithi063.stdout:93 punch from 0x8dfdbaf to 0x8e080f9, (0xa54a bytes) 2024-09-13T09:26:00.141 INFO:teuthology.orchestra.run.smithi063.stdout:94 read 0x8e8f6a8 thru 0x8e97e71 (0x87ca bytes) 2024-09-13T09:26:00.141 INFO:teuthology.orchestra.run.smithi063.stdout:95 read 0x5534160 thru 0x553904c (0x4eed bytes) 2024-09-13T09:26:00.142 INFO:teuthology.orchestra.run.smithi063.stdout:96 read 0x28cd427 thru 0x28d63fb (0x8fd5 bytes) 2024-09-13T09:26:00.142 INFO:teuthology.orchestra.run.smithi063.stdout:97 read 0x1dc7c4e thru 0x1dd0ec0 (0x9273 bytes) 2024-09-13T09:26:00.142 INFO:teuthology.orchestra.run.smithi063.stdout:98 read 0xa32f6e7 thru 0xa33e37f (0xec99 bytes) 2024-09-13T09:26:00.142 INFO:teuthology.orchestra.run.smithi063.stdout:100 writesame 0x508ce7d thru 0x508d3e6 data_size 0x6(0x56a bytes) 2024-09-13T09:26:00.154 INFO:teuthology.orchestra.run.smithi063.stdout:101 punch from 0x2083bea to 0x208b0cc, (0x74e2 bytes) 2024-09-13T09:26:00.163 INFO:teuthology.orchestra.run.smithi063.stdout:103 trunc from 0xd21cba6 to 0x7dca8b3 2024-09-13T09:26:00.200 INFO:teuthology.orchestra.run.smithi063.stdout:104 write 0x805b62d thru 0x805bab8 (0x48c bytes) 2024-09-13T09:26:00.200 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:00.195+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:00.207 INFO:teuthology.orchestra.run.smithi063.stdout:105 read 0x5937763 thru 0x593b87e (0x411c bytes) 2024-09-13T09:26:00.207 INFO:teuthology.orchestra.run.smithi063.stdout:106 writesame 0x152619b thru 0x15332d4 data_size 0x6(0xd13a bytes) 2024-09-13T09:26:00.219 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:00.224 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:26:00.234 INFO:teuthology.orchestra.run.smithi063.stdout:108 write 0x9d878b0 thru 0x9d8c165 (0x48b6 bytes) 2024-09-13T09:26:00.239 INFO:teuthology.orchestra.run.smithi063.stdout:109 read 0x8375798 thru 0x83777cb (0x2034 bytes) 2024-09-13T09:26:00.239 INFO:teuthology.orchestra.run.smithi063.stdout:110 read 0x8f158e thru 0x90122a (0xfc9d bytes) 2024-09-13T09:26:00.240 INFO:teuthology.orchestra.run.smithi063.stdout:112 write 0x285271a thru 0x285cdf4 (0xa6db bytes) 2024-09-13T09:26:00.248 INFO:teuthology.orchestra.run.smithi063.stdout:113 clone 2 order 24 su 2097152 sc 12 2024-09-13T09:26:00.253 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:00.254 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:00.254 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:00.254 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:00.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:00.301 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:00.293+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:00.364 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:00.364 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:00.365 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:00.365 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:00.365 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:00.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:00.402 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:00.397+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:00.449 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:199 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1603797490000001, 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060758599999999998, 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 199 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313896+0000", 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313906+0000", 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1870000000000003e-06 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313920+0000", 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3871e-05 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313940+0000", 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0582999999999999e-05 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.314070+0000", 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013007399999999999 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.314087+0000", 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6413e-05 2024-09-13T09:26:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317280+0000", 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003193244 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317574+0000", 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029457700000000002 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317844+0000", 2024-09-13T09:26:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000269831 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319224+0000", 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001379665 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319823+0000", 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059955700000000002 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319911+0000", 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319974+0000", 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2651999999999996e-05 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:260 3.a 3:55f6faff:::rbd_data.111867511a81.00000000000000eb:head [set-alloc-hint object_size 1048576 write_size 1048576,write 600819~13703] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.016766295, 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002930972, 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 260 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457509+0000", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457561+0000", 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1952e-05 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457595+0000", 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4269e-05 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457620+0000", 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.459e-05 2024-09-13T09:26:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457750+0000", 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013036699999999999 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458002+0000", 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025142099999999999 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458263+0000", 2024-09-13T09:26:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026117700000000002 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.459691+0000", 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014285159999999999 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460298+0000", 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00060671200000000003 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460383+0000", 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4834999999999997e-05 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460443+0000", 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9616e-05 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:262 3.a 3:55f6faff:::rbd_data.111867511a81.00000000000000eb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.490935+0000", 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98334330199999997, 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020802469999999999, 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 262 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490935+0000", 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490935+0000", 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490932+0000", 2024-09-13T09:26:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490947+0000", 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4833e-05 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.490980+0000", 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3442000000000003e-05 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.491014+0000", 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3133000000000003e-05 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.491109+0000", 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5221000000000002e-05 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.491270+0000", 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016113299999999999 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.491433+0000", 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000163263 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.492536+0000", 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011029060000000001 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.492868+0000", 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033192600000000002 2024-09-13T09:26:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.492944+0000", 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6338e-05 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.493015+0000", 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0572000000000001e-05 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91514764999999998, 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:297 3.10 e27/25 ondisk, result = 0)", 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.87329072699999999, 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006064926, 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600986+0000", 2024-09-13T09:26:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600992+0000", 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7699999999999998e-06 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.601009+0000", 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7197000000000002e-05 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.601020+0000", 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0648999999999999e-05 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606853+0000", 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058327409999999998 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606867+0000", 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.43e-05 2024-09-13T09:26:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607052+0000", 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018516600000000001 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:299 3.1e e27/21 3:7b7da939:::rbd_data.111867511a81.0000000000000072:head v 27'2, mlcod=0'0)", 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.611524+0000", 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.86275434699999998, 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011274480000000001, 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611524+0000", 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611524+0000", 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611523+0000", 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611787+0000", 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000264367 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611800+0000", 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2567e-05 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611806+0000", 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.156e-06 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611846+0000", 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0326e-05 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.611860+0000", 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3675e-05 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.612544+0000", 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068461100000000001 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.612651+0000", 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010675699999999999 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:301 3.a 3:51773985:::rbd_data.111867511a81.00000000000000ca:head [set-alloc-hint object_size 1048576 write_size 1048576,write 132677~64267] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.617290+0000", 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.85698843199999997, 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002226352, 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 301 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617290+0000", 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617290+0000", 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617288+0000", 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617430+0000", 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000141743 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617443+0000", 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2996000000000001e-05 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617449+0000", 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.144e-06 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617550+0000", 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100961 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617793+0000", 2024-09-13T09:26:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000243181 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617991+0000", 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019767600000000001 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619410+0000", 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001419093 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619450+0000", 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0034999999999999e-05 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619489+0000", 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8488999999999997e-05 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619516+0000", 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7331e-05 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:324 3.a 3:51773985:::rbd_data.111867511a81.00000000000000ca:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.79721698600000002, 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003713062, 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 324 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677059+0000", 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677071+0000", 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2133000000000001e-05 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677095+0000", 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4015e-05 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677106+0000", 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0542e-05 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677216+0000", 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110306 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677397+0000", 2024-09-13T09:26:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181056 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677562+0000", 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016472500000000001 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678899+0000", 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013370470000000001 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680639+0000", 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001739864 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680712+0000", 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3681000000000001e-05 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680774+0000", 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865000000000005e-05 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:319 3.1e e27/21 3:7b7da939:::rbd_data.111867511a81.0000000000000072:head v 27'3, mlcod=27'1)", 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.678277+0000", 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.79600101400000001, 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001340349, 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678277+0000", 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678277+0000", 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678275+0000", 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678294+0000", 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8836999999999999e-05 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678318+0000", 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3751e-05 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678336+0000", 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8425000000000001e-05 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678422+0000", 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5427999999999996e-05 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678442+0000", 2024-09-13T09:26:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0452999999999999e-05 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.679580+0000", 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001137566 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.679617+0000", 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7737999999999997e-05 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:347 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76183664600000001, 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045695639999999999, 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 347 2024-09-13T09:26:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712439+0000", 2024-09-13T09:26:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712454+0000", 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5228000000000001e-05 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712490+0000", 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5577000000000002e-05 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712517+0000", 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7582000000000001e-05 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712595+0000", 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8011999999999998e-05 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712678+0000", 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2559999999999996e-05 2024-09-13T09:26:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712778+0000", 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9698000000000007e-05 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714327+0000", 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001549784 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716842+0000", 2024-09-13T09:26:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025141489999999998 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716940+0000", 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8374000000000004e-05 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717011+0000", 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1023000000000001e-05 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:344 3.1e e27/21 3:7b3e44d2:::rbd_data.111867511a81.0000000000000027:head v 27'4, mlcod=27'2)", 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.713278+0000", 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.760999545, 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001366319, 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713278+0000", 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713278+0000", 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713277+0000", 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713293+0000", 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6211000000000001e-05 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713619+0000", 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000326616 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713638+0000", 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8672999999999999e-05 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713715+0000", 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6901000000000001e-05 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713733+0000", 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8498000000000001e-05 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714599+0000", 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00086567899999999995 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714645+0000", 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5612000000000003e-05 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:383 3.3 e27/25 3:c7917a04:::rbd_data.111867511a81.000000000000006e:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.786741+0000", 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.68753650799999999, 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00135388, 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786741+0000", 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786741+0000", 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786740+0000", 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786782+0000", 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2546000000000002e-05 2024-09-13T09:26:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786805+0000", 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2861e-05 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786817+0000", 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1267000000000001e-05 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786876+0000", 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9138999999999997e-05 2024-09-13T09:26:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786897+0000", 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1528000000000001e-05 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.788026+0000", 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011291529999999999 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.788095+0000", 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8969999999999999e-05 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:394 3.1e e27/21 3:7e623c36:::rbd_data.111867511a81.000000000000006d:head v 27'5, mlcod=27'3)", 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.819210+0000", 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.65506806500000003, 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017308989999999999, 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819210+0000", 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819210+0000", 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819208+0000", 2024-09-13T09:26:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819271+0000", 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3027999999999994e-05 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819303+0000", 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1965999999999999e-05 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819319+0000", 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6113e-05 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819425+0000", 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010638500000000001 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.819468+0000", 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2613e-05 2024-09-13T09:26:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.820870+0000", 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001402014 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.820941+0000", 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0999000000000002e-05 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:398 3.3 e27/25 3:c7917a04:::rbd_data.111867511a81.000000000000006e:head v 27'2, mlcod=0'0)", 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.839087+0000", 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.63519081099999997, 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017445620000000001, 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839087+0000", 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839087+0000", 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839085+0000", 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839104+0000", 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8193e-05 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839147+0000", 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2945999999999998e-05 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839172+0000", 2024-09-13T09:26:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5803e-05 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839341+0000", 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016850900000000001 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839397+0000", 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6502000000000002e-05 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.840771+0000", 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001373963 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.840832+0000", 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0408999999999997e-05 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:397 3.1e e27/21 3:7e623c36:::rbd_data.111867511a81.000000000000006d:head v 27'6, mlcod=27'4)", 2024-09-13T09:26:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.839294+0000", 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.63498399800000005, 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010275830000000001, 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839294+0000", 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839294+0000", 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839293+0000", 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839309+0000", 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6313000000000001e-05 2024-09-13T09:26:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839344+0000", 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4598999999999999e-05 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839354+0000", 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.026e-05 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839412+0000", 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.855e-05 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.839434+0000", 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1759e-05 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.840247+0000", 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00081231100000000002 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.840322+0000", 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5030999999999994e-05 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:502 3.3 e27/25 3:c6e2c775:::rbd_data.111867511a81.0000000000000096:head v 27'3, mlcod=27'1)", 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.095911+0000", 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.37836721200000001, 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016627370000000001, 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095911+0000", 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095911+0000", 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095909+0000", 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095934+0000", 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5712e-05 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095964+0000", 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9799999999999999e-05 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095980+0000", 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5965999999999999e-05 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.096064+0000", 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3683999999999994e-05 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.096109+0000", 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4823000000000001e-05 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.097454+0000", 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013453339999999999 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.097574+0000", 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011966100000000001 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:518 3.1e e27/21 3:7c1f840a:::rbd_data.111867511a81.00000000000000d0:head v 27'7, mlcod=27'5)", 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.33762986499999997, 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031587289999999999, 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136646+0000", 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136823+0000", 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017679600000000001 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136851+0000", 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8195e-05 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136870+0000", 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.931e-05 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136992+0000", 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121146 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.137050+0000", 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8142e-05 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.139718+0000", 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026680380000000002 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.139807+0000", 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9165000000000004e-05 2024-09-13T09:26:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:522 3.3 e28/25 3:c6e2c775:::rbd_data.111867511a81.0000000000000096:head v 28'4, mlcod=27'2)", 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.168038+0000", 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30624025700000002, 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001152052, 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168038+0000", 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168038+0000", 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168037+0000", 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168068+0000", 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1452999999999999e-05 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168094+0000", 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5281000000000001e-05 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168108+0000", 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168189+0000", 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1200999999999997e-05 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.168212+0000", 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2914000000000001e-05 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.169132+0000", 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00091944699999999999 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.169190+0000", 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7939999999999998e-05 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:524 3.1e e28/21 3:7c1f840a:::rbd_data.111867511a81.00000000000000d0:head v 28'8, mlcod=27'6)", 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.172657+0000", 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30162103400000001, 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093765500000000002, 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172657+0000", 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172657+0000", 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172656+0000", 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172664+0000", 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172699+0000", 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4853e-05 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172715+0000", 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5781999999999999e-05 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172785+0000", 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0259999999999995e-05 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.172811+0000", 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5293999999999999e-05 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.173534+0000", 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072362000000000001 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.173595+0000", 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0303999999999997e-05 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:565 3.10 3:0e125993:::rbd_data.111867511a81.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write 75546~42715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.245179+0000", 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.22909850200000001, 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026099600000000001, 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 565 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245179+0000", 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245179+0000", 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245176+0000", 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245258+0000", 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1635e-05 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245303+0000", 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5448999999999998e-05 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245328+0000", 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4363e-05 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245415+0000", 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7107999999999993e-05 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245541+0000", 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126549 2024-09-13T09:26:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245672+0000", 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000130383 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.246845+0000", 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011733469999999999 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.247675+0000", 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00082975699999999998 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.247744+0000", 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8924000000000006e-05 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.247789+0000", 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5785000000000003e-05 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:00.583 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:00.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:00.584 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:00.577+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:00.591 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:00.618 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:00.640 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:00.659 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:00.679 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:00.684 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:00.677+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:00.702 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:00.725 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:00.742 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:00.743 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:00.746 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:00.748 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:00.748 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:00.748 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:00.748 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:00.748 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:00.749 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:00.749 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:00.749 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:00.753 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:00.786 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:00.778+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:00.860 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:00.860 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:00.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:00.886 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:00.881+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:00.978 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:00.978 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:00.988 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:00.981+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:00.988 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:01.089 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:01.085+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:01.109 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.109 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:01.191 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:01.186+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:01.229 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.229 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.236 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:26:01.241 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:01.291 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:01.285+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:01.348 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone1 from 0x9d8c166 (overlap 0x17db04c) to 0x0 2024-09-13T09:26:01.392 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:01.385+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:01.418 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.418 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.429 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:01.468 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T09:26:01.493 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:01.486+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:01.532 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.532 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.543 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:01.568 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.568 INFO:teuthology.orchestra.run.smithi063.stdout:115 write 0x21bf79f thru 0x21c3a0c (0x426e bytes) 2024-09-13T09:26:01.590 INFO:teuthology.orchestra.run.smithi063.stdout:116 punch from 0x875ec42 to 0x876c6a6, (0xda64 bytes) 2024-09-13T09:26:01.594 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:01.586+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:01.607 INFO:teuthology.orchestra.run.smithi063.stdout:117 write 0xd5ef48f thru 0xd5fac4c (0xb7be bytes) 2024-09-13T09:26:01.616 INFO:teuthology.orchestra.run.smithi063.stdout:118 writesame 0x4cf2ab9 thru 0x4cfbf85 data_size 0x1(0x94cd bytes) 2024-09-13T09:26:01.654 INFO:teuthology.orchestra.run.smithi063.stdout:119 punch from 0xa30eb84 to 0xa3163ef, (0x786b bytes) 2024-09-13T09:26:01.654 INFO:teuthology.orchestra.run.smithi063.stdout:120 read 0x9c072bc thru 0x9c07672 (0x3b7 bytes) 2024-09-13T09:26:01.673 INFO:teuthology.orchestra.run.smithi063.stdout:122 trunc from 0xd5fac4d to 0x2f207c9 2024-09-13T09:26:01.680 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.681 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.692 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:01.727 INFO:teuthology.orchestra.run.smithi063.stdout:123 trunc from 0x2f207c9 to 0x81678a9 2024-09-13T09:26:01.727 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:01.722+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:01.736 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.744 INFO:teuthology.orchestra.run.smithi063.stdout:125 trunc from 0x81678a9 to 0xccb8e32 2024-09-13T09:26:01.753 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.766 INFO:teuthology.orchestra.run.smithi063.stdout:127 write 0xea89d1e thru 0xea92f91 (0x9274 bytes) 2024-09-13T09:26:01.773 INFO:teuthology.orchestra.run.smithi063.stdout:128 writesame 0x7f1f390 thru 0x7f23c78 data_size 0x1(0x48e9 bytes) 2024-09-13T09:26:01.790 INFO:teuthology.orchestra.run.smithi063.stdout:130 read 0x2f8f5ab thru 0x2f97d07 (0x875d bytes) 2024-09-13T09:26:01.791 INFO:teuthology.orchestra.run.smithi063.stdout:131 read 0x6926424 thru 0x69272b4 (0xe91 bytes) 2024-09-13T09:26:01.792 INFO:teuthology.orchestra.run.smithi063.stdout:132 read 0x6423c6e thru 0x6425436 (0x17c9 bytes) 2024-09-13T09:26:01.792 INFO:teuthology.orchestra.run.smithi063.stdout:133 write 0xe9fe521 thru 0xea0954e (0xb02e bytes) 2024-09-13T09:26:01.795 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:01.789+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:01.799 INFO:teuthology.orchestra.run.smithi063.stdout:135 read 0x846451e thru 0x8468285 (0x3d68 bytes) 2024-09-13T09:26:01.800 INFO:teuthology.orchestra.run.smithi063.stdout:137 write 0x7130cc thru 0x714929 (0x185e bytes) 2024-09-13T09:26:01.821 INFO:teuthology.orchestra.run.smithi063.stdout:138 trunc from 0xea92f92 to 0x4b0a400 2024-09-13T09:26:01.845 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.845 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.857 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:139 write 0xe72a8bd thru 0xe7360e8 (0xb82c bytes) 2024-09-13T09:26:01.879 INFO:teuthology.orchestra.run.smithi063.stdout:141 write 0xbbbf36f thru 0xbbc6b62 (0x77f4 bytes) 2024-09-13T09:26:01.896 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:01.889+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:01.901 INFO:teuthology.orchestra.run.smithi063.stdout:143 trunc from 0xe7360e9 to 0x3864e30 2024-09-13T09:26:01.930 INFO:teuthology.orchestra.run.smithi063.stdout:144 read 0x4cec83 thru 0x4d85f0 (0x996e bytes) 2024-09-13T09:26:01.931 INFO:teuthology.orchestra.run.smithi063.stdout:145 read 0x12bb9f9 thru 0x12c8b15 (0xd11d bytes) 2024-09-13T09:26:01.932 INFO:teuthology.orchestra.run.smithi063.stdout:146 read 0xadef32 thru 0xaeba30 (0xcaff bytes) 2024-09-13T09:26:01.933 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0xeb6262d 2024-09-13T09:26:01.951 INFO:teuthology.orchestra.run.smithi063.stdout:147 trunc from 0x3864e30 to 0xeb6262d 2024-09-13T09:26:01.960 INFO:teuthology.orchestra.run.smithi063.stdout:148 read 0x1b24698 thru 0x1b2d6f5 (0x905e bytes) 2024-09-13T09:26:01.961 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.961 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.961 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.961 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.961 INFO:teuthology.orchestra.run.smithi063.stdout:154 read 0x3100030 thru 0x310f422 (0xf3f3 bytes) 2024-09-13T09:26:01.962 INFO:teuthology.orchestra.run.smithi063.stdout:155 read 0x6f672ff thru 0x6f6a5ef (0x32f1 bytes) 2024-09-13T09:26:01.963 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.963 INFO:teuthology.orchestra.run.smithi063.stdout:157 write 0xd6d4818 thru 0xd6dfd41 (0xb52a bytes) 2024-09-13T09:26:01.970 INFO:teuthology.orchestra.run.smithi063.stdout:158 read 0xcff7af7 thru 0xd002662 (0xab6c bytes) 2024-09-13T09:26:01.970 INFO:teuthology.orchestra.run.smithi063.stdout:159 read 0x2c51983 thru 0x2c5d919 (0xbf97 bytes) 2024-09-13T09:26:01.971 INFO:teuthology.orchestra.run.smithi063.stdout:161 punch from 0xc2d16eb to 0xc2e1057, (0xf96c bytes) 2024-09-13T09:26:01.971 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.971 INFO:teuthology.orchestra.run.smithi063.stdout:164 write 0x4cf0433 thru 0x4cf0a93 (0x661 bytes) 2024-09-13T09:26:01.974 INFO:teuthology.orchestra.run.smithi063.stdout:166 write 0xa292bff thru 0xa29884a (0x5c4c bytes) 2024-09-13T09:26:01.977 INFO:teuthology.orchestra.run.smithi063.stdout:167 write 0x1a6920d thru 0x1a6e8a2 (0x5696 bytes) 2024-09-13T09:26:01.980 INFO:teuthology.orchestra.run.smithi063.stdout:169 read 0x87cd2eb thru 0x87d0ed9 (0x3bef bytes) 2024-09-13T09:26:01.981 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.981 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.981 INFO:teuthology.orchestra.run.smithi063.stdout:172 read 0xb7372b thru 0xb7c97a (0x9250 bytes) 2024-09-13T09:26:01.982 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:01.982 INFO:teuthology.orchestra.run.smithi063.stdout:175 punch from 0xd3538d1 to 0xd361fce, (0xe6fd bytes) 2024-09-13T09:26:01.982 INFO:teuthology.orchestra.run.smithi063.stdout:178 read 0xbd59f5e thru 0xbd5ef2f (0x4fd2 bytes) 2024-09-13T09:26:01.982 INFO:teuthology.orchestra.run.smithi063.stdout:179 write 0x714b2e3 thru 0x71538e8 (0x8606 bytes) 2024-09-13T09:26:01.986 INFO:teuthology.orchestra.run.smithi063.stdout:180 read 0x83f69f6 thru 0x83f80a8 (0x16b3 bytes) 2024-09-13T09:26:01.986 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:01.986 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:01.996 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:01.989+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:01.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:02.000 INFO:teuthology.orchestra.run.smithi063.stdout:183 trunc from 0xeb6262d to 0x7069e70 2024-09-13T09:26:02.028 INFO:teuthology.orchestra.run.smithi063.stdout:184 read 0x4666e50 thru 0x466f503 (0x86b4 bytes) 2024-09-13T09:26:02.029 INFO:teuthology.orchestra.run.smithi063.stdout:185 writesame 0x534f2eb thru 0x53519a9 data_size 0xd(0x26bf bytes) 2024-09-13T09:26:02.031 INFO:teuthology.orchestra.run.smithi063.stdout:186 read 0x4785e75 thru 0x478694a (0xad6 bytes) 2024-09-13T09:26:02.032 INFO:teuthology.orchestra.run.smithi063.stdout:187 clone 3 order 18 su 32768 sc 14 2024-09-13T09:26:02.097 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:02.093+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:02.103 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.103 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.113 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:02.198 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:02.197+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:02.212 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:26:02.212 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:02.212 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T09:26:02.212 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:26:02.212 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:26:02.213 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T09:26:02.275 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.275 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:02.299 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:02.293+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:02.337 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:02.337 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:26:02.337 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:02.352 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone2 from 0x7069e70 (overlap 0x2f207c9) to 0x2a6a09b 2024-09-13T09:26:02.395 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.395 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.400 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:02.393+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:02.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:02.494 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T09:26:02.500 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:02.494+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:02.508 INFO:teuthology.orchestra.run.smithi063.stdout:188 read 0x5e947c7 thru 0x5e95c57 (0x1491 bytes) 2024-09-13T09:26:02.516 INFO:teuthology.orchestra.run.smithi063.stdout:189 write 0x62629e thru 0x62f67d (0x93e0 bytes) 2024-09-13T09:26:02.524 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.524 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.528 INFO:teuthology.orchestra.run.smithi063.stdout:191 writesame 0x4009846 thru 0x4015aab data_size 0x2(0xc266 bytes) 2024-09-13T09:26:02.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:02.563 INFO:teuthology.orchestra.run.smithi063.stdout:192 write 0xad6fa14 thru 0xad7dd6d (0xe35a bytes) 2024-09-13T09:26:02.575 INFO:teuthology.orchestra.run.smithi063.stdout:193 read 0x7212fcc thru 0x721e667 (0xb69c bytes) 2024-09-13T09:26:02.577 INFO:teuthology.orchestra.run.smithi063.stdout:194 trunc from 0xad7dd6e to 0xbbbf7d8 2024-09-13T09:26:02.594 INFO:teuthology.orchestra.run.smithi063.stdout:197 write 0xbbe2ef9 thru 0xbbf0500 (0xd608 bytes) 2024-09-13T09:26:02.601 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:02.594+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:02.603 INFO:teuthology.orchestra.run.smithi063.stdout:198 punch from 0x3bd7617 to 0x3bd9e96, (0x287f bytes) 2024-09-13T09:26:02.604 INFO:teuthology.orchestra.run.smithi063.stdout:199 writesame 0xa01c0b thru 0xa0b6fb data_size 0x1(0x9af1 bytes) 2024-09-13T09:26:02.646 INFO:teuthology.orchestra.run.smithi063.stdout:200 trunc from 0xbbf0501 to 0x33d8035 2024-09-13T09:26:02.687 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.687 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.694 INFO:teuthology.orchestra.run.smithi063.stdout:201 punch from 0x1f0cf to 0x2f0ca, (0xfffb bytes) 2024-09-13T09:26:02.694 INFO:teuthology.orchestra.run.smithi063.stdout:202 punch from 0x2fc98f0 to 0x2fcd9a2, (0x40b2 bytes) 2024-09-13T09:26:02.694 INFO:teuthology.orchestra.run.smithi063.stdout:203 read 0x30ece08 thru 0x30fb30a (0xe503 bytes) 2024-09-13T09:26:02.696 INFO:teuthology.orchestra.run.smithi063.stdout:204 read 0xa91c3 thru 0xab8b2 (0x26f0 bytes) 2024-09-13T09:26:02.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:02.712 INFO:teuthology.orchestra.run.smithi063.stdout:207 trunc from 0x33d8035 to 0xbe3e0f9 2024-09-13T09:26:02.712 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:02.706+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:02.730 INFO:teuthology.orchestra.run.smithi063.stdout:208 writesame 0xc956d29 thru 0xc960170 data_size 0x28(0x9448 bytes) 2024-09-13T09:26:02.731 INFO:teuthology.orchestra.run.smithi063.stdout:210 read 0xad3290e thru 0xad34a26 (0x2119 bytes) 2024-09-13T09:26:02.731 INFO:teuthology.orchestra.run.smithi063.stdout:211 writesame 0x2418c2b thru 0x241d26a data_size 0x20(0x4640 bytes) 2024-09-13T09:26:02.756 INFO:teuthology.orchestra.run.smithi063.stdout:212 write 0xea25f05 thru 0xea2ee73 (0x8f6f bytes) 2024-09-13T09:26:02.761 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:02.768 INFO:teuthology.orchestra.run.smithi063.stdout:213 punch from 0x7ca5c70 to 0x7ca60e2, (0x472 bytes) 2024-09-13T09:26:02.768 INFO:teuthology.orchestra.run.smithi063.stdout:215 read 0xa6901f1 thru 0xa6902de (0xee bytes) 2024-09-13T09:26:02.768 INFO:teuthology.orchestra.run.smithi063.stdout:216 read 0x685964b thru 0x68608bc (0x7272 bytes) 2024-09-13T09:26:02.768 INFO:teuthology.orchestra.run.smithi063.stdout:217 read 0x709de2a thru 0x70a1fa7 (0x417e bytes) 2024-09-13T09:26:02.768 INFO:teuthology.orchestra.run.smithi063.stdout:218 write 0xd10b41a thru 0xd117b74 (0xc75b bytes) 2024-09-13T09:26:02.776 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:02.776 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:02.776 INFO:teuthology.orchestra.run.smithi063.stdout:223 read 0xa7933eb thru 0xa7950ec (0x1d02 bytes) 2024-09-13T09:26:02.777 INFO:teuthology.orchestra.run.smithi063.stdout:225 writesame 0x3abff26 thru 0x3ac79b6 data_size 0x1(0x7a91 bytes) 2024-09-13T09:26:02.802 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:02.798+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:02.803 INFO:teuthology.orchestra.run.smithi063.stdout:226 read 0x4ee37df thru 0x4ee62a4 (0x2ac6 bytes) 2024-09-13T09:26:02.803 INFO:teuthology.orchestra.run.smithi063.stdout:228 writesame 0xa09c8d1 thru 0xa0a1ae0 data_size 0x65(0x5210 bytes) 2024-09-13T09:26:02.812 INFO:teuthology.orchestra.run.smithi063.stdout:229 read 0x912571a thru 0x912db8e (0x8475 bytes) 2024-09-13T09:26:02.812 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.812 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:02.826 INFO:teuthology.orchestra.run.smithi063.stdout:230 trunc from 0xea2ee74 to 0x88eaf46 2024-09-13T09:26:02.850 INFO:teuthology.orchestra.run.smithi063.stdout:231 trunc from 0x88eaf46 to 0x9a26a4b 2024-09-13T09:26:02.867 INFO:teuthology.orchestra.run.smithi063.stdout:233 trunc from 0x9a26a4b to 0x59f49c5 2024-09-13T09:26:02.897 INFO:teuthology.orchestra.run.smithi063.stdout:234 writesame 0xa47ca54 thru 0xa483992 data_size 0xb(0x6f3f bytes) 2024-09-13T09:26:02.897 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:02.898 INFO:teuthology.orchestra.run.smithi063.stdout:236 punch from 0x3890c99 to 0x38946e9, (0x3a50 bytes) 2024-09-13T09:26:02.903 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:02.897+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:02.911 INFO:teuthology.orchestra.run.smithi063.stdout:237 write 0xc7efee6 thru 0xc7ffdca (0xfee5 bytes) 2024-09-13T09:26:02.917 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:02.924 INFO:teuthology.orchestra.run.smithi063.stdout:238 read 0xb0f3fc9 thru 0xb0f4376 (0x3ae bytes) 2024-09-13T09:26:02.924 INFO:teuthology.orchestra.run.smithi063.stdout:239 writesame 0x8bc3c90 thru 0x8bc6490 data_size 0x31(0x2801 bytes) 2024-09-13T09:26:02.928 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:02.928 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:02.929 INFO:teuthology.orchestra.run.smithi063.stdout:240 read 0x9d55f90 thru 0x9d5aae9 (0x4b5a bytes) 2024-09-13T09:26:02.929 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:02.929 INFO:teuthology.orchestra.run.smithi063.stdout:244 read 0x8a8a1af thru 0x8a8c6ae (0x2500 bytes) 2024-09-13T09:26:02.929 INFO:teuthology.orchestra.run.smithi063.stdout:245 read 0x23c656c thru 0x23ce1ba (0x7c4f bytes) 2024-09-13T09:26:02.937 INFO:teuthology.orchestra.run.smithi063.stdout:246 trunc from 0xc7ffdcb to 0x92ba4e5 2024-09-13T09:26:02.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:02.941 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:02.958 INFO:teuthology.orchestra.run.smithi063.stdout:247 write 0xe7a25f thru 0xe7c994 (0x2736 bytes) 2024-09-13T09:26:02.966 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:02.966 INFO:teuthology.orchestra.run.smithi063.stdout:249 read 0x16ae4e thru 0x171ca2 (0x6e55 bytes) 2024-09-13T09:26:02.967 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:02.968 INFO:teuthology.orchestra.run.smithi063.stdout:250 write 0x7d960 thru 0x850cb (0x776c bytes) 2024-09-13T09:26:02.982 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:02.991 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:02.996 INFO:teuthology.orchestra.run.smithi063.stdout:252 write 0xae939bf thru 0xaea0427 (0xca69 bytes) 2024-09-13T09:26:03.004 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:02.998+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:03.014 INFO:teuthology.orchestra.run.smithi063.stdout:253 trunc from 0xaea0428 to 0x6eca2e7 2024-09-13T09:26:03.018 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:03.033 INFO:teuthology.orchestra.run.smithi063.stdout:254 read 0x13ec1cc thru 0x13f7a3d (0xb872 bytes) 2024-09-13T09:26:03.047 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:03.047 INFO:teuthology.orchestra.run.smithi063.stdout:257 trunc from 0x6eca2e7 to 0xd425739 2024-09-13T09:26:03.054 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:54.169985+0000", 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8816574549999991, 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040221709999999997, 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 7 2024-09-13T09:26:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169985+0000", 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169985+0000", 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169983+0000", 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.169996+0000", 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2428e-05 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170026+0000", 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0026e-05 2024-09-13T09:26:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170045+0000", 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9235000000000001e-05 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170160+0000", 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011499 2024-09-13T09:26:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170314+0000", 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015367199999999999 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.170444+0000", 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013062999999999999 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.171457+0000", 2024-09-13T09:26:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010129080000000001 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.173783+0000", 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023255279999999999 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.173905+0000", 2024-09-13T09:26:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122707 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:54.174007+0000", 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010195 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:55.191077+0000", 2024-09-13T09:26:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.8605652629999998, 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040530949999999996, 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191077+0000", 2024-09-13T09:26:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191077+0000", 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191075+0000", 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191088+0000", 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2452e-05 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191148+0000", 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886999999999998e-05 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191171+0000", 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3524e-05 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191305+0000", 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133714 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191561+0000", 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025581399999999999 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.191944+0000", 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038268500000000001 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.194062+0000", 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021186960000000002 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.195002+0000", 2024-09-13T09:26:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093997999999999998 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.195072+0000", 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9669e-05 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:55.195130+0000", 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:249 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'5, mlcod=27'3)", 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.440549+0000", 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6110935939999997, 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004070756, 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440549+0000", 2024-09-13T09:26:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440549+0000", 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440548+0000", 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440561+0000", 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3890000000000001e-05 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440586+0000", 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4286000000000001e-05 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440598+0000", 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2276000000000001e-05 2024-09-13T09:26:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440702+0000", 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103905 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.440774+0000", 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1871000000000004e-05 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444541+0000", 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037672560000000001 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.444620+0000", 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8794999999999994e-05 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:265 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'10, mlcod=27'7)", 2024-09-13T09:26:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.498956+0000", 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5526863020000001, 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003019861, 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498956+0000", 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498956+0000", 2024-09-13T09:26:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498955+0000", 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.498992+0000", 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7159000000000001e-05 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499013+0000", 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1294e-05 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499027+0000", 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3188e-05 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499128+0000", 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010173100000000001 2024-09-13T09:26:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499179+0000", 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1147999999999999e-05 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.501945+0000", 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002765127 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.501976+0000", 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1681999999999999e-05 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:266 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'11, mlcod=27'7)", 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.499878+0000", 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5517643109999999, 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026630769999999998, 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499878+0000", 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499878+0000", 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499877+0000", 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499887+0000", 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0057999999999999e-05 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499901+0000", 2024-09-13T09:26:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3831e-05 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499911+0000", 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7180000000000008e-06 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.499992+0000", 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0711999999999996e-05 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.500027+0000", 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5639999999999998e-05 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.502501+0000", 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024737779999999998 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.502541+0000", 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0315999999999998e-05 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:304 3.1 3:8457a7c9:::rbd_data.111867511a81.000000000000003b:head [set-alloc-hint object_size 1048576 write_size 1048576,write 745980~40452] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.624278+0000", 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4273644619999999, 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033338209999999998, 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 304 2024-09-13T09:26:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624278+0000", 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624278+0000", 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624276+0000", 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624425+0000", 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014859000000000001 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624450+0000", 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4714000000000001e-05 2024-09-13T09:26:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624462+0000", 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2816999999999999e-05 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624538+0000", 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5776000000000001e-05 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624672+0000", 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133969 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624799+0000", 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012640700000000001 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.626515+0000", 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001716312 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627471+0000", 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000956514 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627552+0000", 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0278000000000006e-05 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627612+0000", 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0356999999999998e-05 2024-09-13T09:26:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:305 3.14 3:2c215406:::rbd_data.111867511a81.000000000000003c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~14223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.624899+0000", 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.426743959, 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028360960000000002, 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 305 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624899+0000", 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624899+0000", 2024-09-13T09:26:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624897+0000", 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624926+0000", 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9343e-05 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624972+0000", 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5799e-05 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.624989+0000", 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6786000000000002e-05 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.625076+0000", 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7101000000000005e-05 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.625332+0000", 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025624099999999999 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.625517+0000", 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000185024 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627419+0000", 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019023 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627656+0000", 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023713099999999999 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627702+0000", 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6063000000000003e-05 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.627735+0000", 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2378000000000001e-05 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:307 3.9 3:92c4c1be:::rbd_data.111867511a81.0000000000000087:head [set-alloc-hint object_size 1048576 write_size 1048576,write 200646~1748] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.631747+0000", 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4198955099999999, 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030287790000000001, 2024-09-13T09:26:03.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 307 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631747+0000", 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631747+0000", 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631746+0000", 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631758+0000", 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2132e-05 2024-09-13T09:26:03.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631782+0000", 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3838000000000001e-05 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631794+0000", 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2054e-05 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631844+0000", 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0912000000000003e-05 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.631957+0000", 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011254700000000001 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.632037+0000", 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9686999999999998e-05 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.632771+0000", 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00073456599999999997 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.634707+0000", 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001935453 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.634750+0000", 2024-09-13T09:26:03.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3723000000000001e-05 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.634776+0000", 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5490000000000002e-05 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:311 3.1a 3:5dde7a26:::rbd_data.111867511a81.00000000000000af:head [set-alloc-hint object_size 1048576 write_size 1048576,write 780118~6314] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.638311+0000", 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.4133318689999999, 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032026110000000002, 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 311 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638311+0000", 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638311+0000", 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638310+0000", 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638319+0000", 2024-09-13T09:26:03.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4380000000000001e-06 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638332+0000", 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2883e-05 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638338+0000", 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2430000000000002e-06 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638388+0000", 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9635999999999997e-05 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638506+0000", 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118208 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.638633+0000", 2024-09-13T09:26:03.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012671500000000001 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.639506+0000", 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087312100000000003 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.641402+0000", 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018954810000000001 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.641466+0000", 2024-09-13T09:26:03.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4444999999999997e-05 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.641513+0000", 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7448999999999999e-05 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:321 3.9 3:92c4c1be:::rbd_data.111867511a81.0000000000000087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.677117+0000", 2024-09-13T09:26:03.108 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3745261420000001, 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032764259999999998, 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 321 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677117+0000", 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677117+0000", 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677115+0000", 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677127+0000", 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2186e-05 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677160+0000", 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2990000000000001e-05 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677178+0000", 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8229000000000001e-05 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677292+0000", 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113631 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677466+0000", 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017401600000000001 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677698+0000", 2024-09-13T09:26:03.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000232432 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.679621+0000", 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019230320000000001 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680214+0000", 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059296900000000005 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680310+0000", 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6212000000000007e-05 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680393+0000", 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2720999999999996e-05 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:323 3.1a 3:5dde7a26:::rbd_data.111867511a81.00000000000000af:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.677236+0000", 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3744070740000001, 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003128732, 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 323 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677236+0000", 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677236+0000", 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677234+0000", 2024-09-13T09:26:03.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677243+0000", 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5909999999999996e-06 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677260+0000", 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6750999999999999e-05 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677276+0000", 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677342+0000", 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6755999999999996e-05 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677422+0000", 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9067999999999996e-05 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677524+0000", 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102769 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.679635+0000", 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021109660000000001 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680180+0000", 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054472799999999997 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680289+0000", 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109337 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680364+0000", 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4960000000000001e-05 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:345 3.1 3:8457a7c9:::rbd_data.111867511a81.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.120 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712480+0000", 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3391630270000001, 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036688789999999999, 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 345 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712480+0000", 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712480+0000", 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712477+0000", 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712489+0000", 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2214e-05 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712522+0000", 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2830999999999998e-05 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712540+0000", 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7584999999999999e-05 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712622+0000", 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2041999999999998e-05 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712781+0000", 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015888799999999999 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712976+0000", 2024-09-13T09:26:03.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000195484 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714452+0000", 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014758690000000001 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716101+0000", 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001648526 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716128+0000", 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7078999999999999e-05 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716149+0000", 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0744e-05 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:346 3.14 3:2c215406:::rbd_data.111867511a81.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712572+0000", 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.339070703, 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043335309999999998, 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 346 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712572+0000", 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712572+0000", 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712571+0000", 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712578+0000", 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3599999999999998e-06 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712593+0000", 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4948e-05 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712605+0000", 2024-09-13T09:26:03.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1837e-05 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712710+0000", 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104838 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712914+0000", 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020453499999999999 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713171+0000", 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025680899999999999 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714838+0000", 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001666596 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716751+0000", 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001913355 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716851+0000", 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9704000000000006e-05 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716905+0000", 2024-09-13T09:26:03.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4620999999999998e-05 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:340 3.6 e27/25 3:66b996d9:::rbd_data.111867511a81.0000000000000023:head v 27'6, mlcod=27'4)", 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.713909+0000", 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.337734164, 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028999020000000002, 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713909+0000", 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713909+0000", 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713908+0000", 2024-09-13T09:26:03.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713915+0000", 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2139999999999999e-06 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713927+0000", 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1922e-05 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.713934+0000", 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1339999999999998e-06 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714029+0000", 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4649999999999997e-05 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714043+0000", 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3936e-05 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716784+0000", 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027417330000000001 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716808+0000", 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4145999999999999e-05 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:383 3.3 3:c7917a04:::rbd_data.111867511a81.000000000000006e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 176515~22423] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.785969+0000", 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.265673821, 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002457791, 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 383 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.785969+0000", 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.785969+0000", 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.785967+0000", 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786000+0000", 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3099000000000003e-05 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786026+0000", 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.569e-05 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786039+0000", 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3540000000000001e-05 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786141+0000", 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010212 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786378+0000", 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023631800000000001 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.786579+0000", 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020129600000000001 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.787957+0000", 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013779490000000001 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.788302+0000", 2024-09-13T09:26:03.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000345414 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.788359+0000", 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7194000000000002e-05 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.788427+0000", 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7161000000000003e-05 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:398 3.3 3:c7917a04:::rbd_data.111867511a81.000000000000006e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.838635+0000", 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.2130076440000002, 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002519028, 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 398 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838635+0000", 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838635+0000", 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838632+0000", 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838648+0000", 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6469000000000001e-05 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838682+0000", 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3633000000000001e-05 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838699+0000", 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7316000000000002e-05 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838760+0000", 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0531e-05 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838854+0000", 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4415999999999999e-05 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.838959+0000", 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010450099999999999 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.840080+0000", 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001121578 2024-09-13T09:26:03.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.841019+0000", 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000938893 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.841100+0000", 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0334999999999996e-05 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.841154+0000", 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4469000000000003e-05 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:461 3.1a 3:5cc0ca70:::rbd_data.111867511a81.000000000000004c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.016057+0000", 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.0355856320000001, 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014701226, 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 461 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016057+0000", 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016057+0000", 2024-09-13T09:26:03.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016056+0000", 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016062+0000", 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7130000000000002e-06 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016074+0000", 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1764e-05 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016080+0000", 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016192+0000", 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112063 2024-09-13T09:26:03.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016573+0000", 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038042799999999998 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.020585+0000", 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040121310000000004 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030630+0000", 2024-09-13T09:26:03.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01004496 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030648+0000", 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8096000000000001e-05 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030712+0000", 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4058000000000002e-05 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030758+0000", 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6487999999999999e-05 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:502 3.3 3:c6e2c775:::rbd_data.111867511a81.0000000000000096:head [set-alloc-hint object_size 1048576 write_size 1048576,write 780904~5528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.095069+0000", 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.956574013, 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029474940000000002, 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 502 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095069+0000", 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095069+0000", 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095067+0000", 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095087+0000", 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0406000000000001e-05 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095120+0000", 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3336000000000001e-05 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095144+0000", 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3896e-05 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095247+0000", 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102581 2024-09-13T09:26:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095445+0000", 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019851600000000001 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.095714+0000", 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000268767 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.097344+0000", 2024-09-13T09:26:03.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016302070000000001 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.097857+0000", 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00051257500000000003 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.097963+0000", 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010566200000000001 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.098016+0000", 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3528e-05 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:519 3.1c e27/21 3:3fda0657:::rbd_object_map.111867511a81:head v 27'69, mlcod=27'67)", 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.142641+0000", 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.9090019859999998, 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039619829999999997, 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142641+0000", 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142641+0000", 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142640+0000", 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142649+0000", 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7569999999999999e-06 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142666+0000", 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6592e-05 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142674+0000", 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3100000000000001e-06 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142869+0000", 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019457900000000001 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.142919+0000", 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9777999999999997e-05 2024-09-13T09:26:03.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.146528+0000", 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003609798 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.146603+0000", 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4260999999999999e-05 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:520 3.e e27/21 3:74ab7c77:::rbd_data.111867511a81.0000000000000052:head v 27'2, mlcod=0'0)", 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.148409+0000", 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.9032337369999999, 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042001190000000004, 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148409+0000", 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148409+0000", 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148407+0000", 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148424+0000", 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6671e-05 2024-09-13T09:26:03.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148448+0000", 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4250999999999999e-05 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148460+0000", 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1399999999999999e-05 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148600+0000", 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140152 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.148629+0000", 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9320999999999999e-05 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.152535+0000", 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039053640000000001 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.152609+0000", 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4543999999999996e-05 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.171 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:03.172 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.172 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:03.173 INFO:teuthology.orchestra.run.smithi063.stdout:260 trunc from 0xd425739 to 0x4dbbe7 2024-09-13T09:26:03.173 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:03.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:03.175 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:03.176 INFO:teuthology.orchestra.run.smithi063.stdout:261 write 0x471f432 thru 0x47260e7 (0x6cb6 bytes) 2024-09-13T09:26:03.176 INFO:teuthology.orchestra.run.smithi063.stdout:262 write 0x5e26f48 thru 0x5e2ba17 (0x4ad0 bytes) 2024-09-13T09:26:03.176 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:03.166+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:03.196 INFO:teuthology.orchestra.run.smithi063.stdout:263 trunc from 0x5e2ba18 to 0x12530b 2024-09-13T09:26:03.249 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:26:03.274 INFO:teuthology.orchestra.run.smithi063.stdout:264 write 0x5e6e3d4 thru 0x5e7250b (0x4138 bytes) 2024-09-13T09:26:03.274 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:03.270+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:03.292 INFO:teuthology.orchestra.run.smithi063.stdout:265 trunc from 0x5e7250c to 0xd259c92 2024-09-13T09:26:03.300 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.300 INFO:teuthology.orchestra.run.smithi063.stdout:270 punch from 0xbe274bf to 0xbe296d7, (0x2218 bytes) 2024-09-13T09:26:03.300 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.300 INFO:teuthology.orchestra.run.smithi063.stdout:272 write 0xa69dc41 thru 0xa6a222a (0x45ea bytes) 2024-09-13T09:26:03.301 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:03.301 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:03.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2179 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.300284+0000", 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00070215800000000004, 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074746700000000003, 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2179 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300284+0000", 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300284+0000", 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300282+0000", 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300295+0000", 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2883e-05 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300326+0000", 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1168999999999999e-05 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300340+0000", 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3944000000000001e-05 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300440+0000", 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100134 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300606+0000", 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000165196 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2180 3.c 3.e31f880c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.300361+0000", 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.000625839, 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077514900000000004, 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:03.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2180 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300361+0000", 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300361+0000", 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300360+0000", 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300365+0000", 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6899999999999997e-06 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300390+0000", 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4973e-05 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.300398+0000", 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5959999999999998e-06 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.309 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:26:03.310 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 2 2024-09-13T09:26:03.310 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:03.310 INFO:teuthology.orchestra.run.smithi063.stdout:273 write 0x1bf77af thru 0x1c04ea1 (0xd6f3 bytes) 2024-09-13T09:26:03.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:03.319 INFO:teuthology.orchestra.run.smithi063.stdout:274 write 0xab20446 thru 0xab25baa (0x5765 bytes) 2024-09-13T09:26:03.324 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.324 INFO:teuthology.orchestra.run.smithi063.stdout:276 trunc from 0xd259c92 to 0xd4bc434 2024-09-13T09:26:03.335 INFO:teuthology.orchestra.run.smithi063.stdout:277 write 0xe2f737 thru 0xe38b83 (0x944d bytes) 2024-09-13T09:26:03.343 INFO:teuthology.orchestra.run.smithi063.stdout:279 read 0x5883760 thru 0x588e3c4 (0xac65 bytes) 2024-09-13T09:26:03.343 INFO:teuthology.orchestra.run.smithi063.stdout:281 read 0xa95c726 thru 0xa96219e (0x5a79 bytes) 2024-09-13T09:26:03.365 INFO:teuthology.orchestra.run.smithi063.stdout:284 trunc from 0xd4bc434 to 0x9cd322 2024-09-13T09:26:03.374 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:03.370+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:03.385 INFO:teuthology.orchestra.run.smithi063.stdout:285 read 0x326e78 thru 0x3351cc (0xe355 bytes) 2024-09-13T09:26:03.416 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:03.416 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:03.418 INFO:teuthology.orchestra.run.smithi063.stdout:287 writesame 0xe230ce7 thru 0xe230d9e data_size 0x4(0xb8 bytes) 2024-09-13T09:26:03.418 INFO:teuthology.orchestra.run.smithi063.stdout:288 read 0x4fe5e1 thru 0x507edd (0x98fd bytes) 2024-09-13T09:26:03.418 INFO:teuthology.orchestra.run.smithi063.stdout:289 write 0x2ff184a thru 0x2ffa5e5 (0x8d9c bytes) 2024-09-13T09:26:03.425 INFO:teuthology.orchestra.run.smithi063.stdout:291 read 0xbf587ac thru 0xbf5d749 (0x4f9e bytes) 2024-09-13T09:26:03.425 INFO:teuthology.orchestra.run.smithi063.stdout:293 writesame 0x4505e8d thru 0x45108ca data_size 0x4d(0xaa3e bytes) 2024-09-13T09:26:03.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:03.436 INFO:teuthology.orchestra.run.smithi063.stdout:294 writesame 0x316a11 thru 0x3203b4 data_size 0x2(0x99a4 bytes) 2024-09-13T09:26:03.453 INFO:teuthology.orchestra.run.smithi063.stdout:296 write 0x52a8c0e thru 0x52aa589 (0x197c bytes) 2024-09-13T09:26:03.461 INFO:teuthology.orchestra.run.smithi063.stdout:297 writesame 0x7551ae7 thru 0x755c1c1 data_size 0x1(0xa6db bytes) 2024-09-13T09:26:03.475 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:03.469+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:03.482 INFO:teuthology.orchestra.run.smithi063.stdout:298 read 0xd4c00f5 thru 0xd4c597f (0x588b bytes) 2024-09-13T09:26:03.482 INFO:teuthology.orchestra.run.smithi063.stdout:300 read 0x4b14d2f thru 0x4b1dd27 (0x8ff9 bytes) 2024-09-13T09:26:03.482 INFO:teuthology.orchestra.run.smithi063.stdout:303 read 0x960c113 thru 0x960cd47 (0xc35 bytes) 2024-09-13T09:26:03.483 INFO:teuthology.orchestra.run.smithi063.stdout:304 write 0x2898cf2 thru 0x28992ce (0x5dd bytes) 2024-09-13T09:26:03.488 INFO:teuthology.orchestra.run.smithi063.stdout:307 writesame 0x65cb08d thru 0x65d8e75 data_size 0x1(0xdde9 bytes) 2024-09-13T09:26:03.518 INFO:teuthology.orchestra.run.smithi063.stdout:308 writesame 0x24ef306 thru 0x24f8b9a data_size 0x35(0x9895 bytes) 2024-09-13T09:26:03.526 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0xeda7706 2024-09-13T09:26:03.528 INFO:teuthology.orchestra.run.smithi063.stdout:309 trunc from 0xe230d9f to 0xeda7706 2024-09-13T09:26:03.536 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:03.536 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:270 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0351674729999996, 2024-09-13T09:26:03.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087584770000000006, 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 270 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.538 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497696+0000", 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497707+0000", 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1214e-05 2024-09-13T09:26:03.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497716+0000", 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0250000000000008e-06 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497723+0000", 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:26:03.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504669+0000", 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069460399999999997 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504713+0000", 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3582999999999999e-05 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504887+0000", 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174196 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505909+0000", 2024-09-13T09:26:03.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001022565 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506362+0000", 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045264300000000002 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506415+0000", 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2852000000000002e-05 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506455+0000", 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0556000000000003e-05 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1851 3.1d 3:bdd77b2d:::rbd_data.1118e3e5e25d.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.44970843999999999, 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084777419999999999, 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1851 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083154+0000", 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083170+0000", 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6062e-05 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083198+0000", 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8191000000000002e-05 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083222+0000", 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083391+0000", 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000169276 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083553+0000", 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016206400000000001 2024-09-13T09:26:03.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083765+0000", 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021140699999999999 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.085517+0000", 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001752691 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091479+0000", 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059616670000000004 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091557+0000", 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8090999999999995e-05 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091634+0000", 2024-09-13T09:26:03.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6495999999999994e-05 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1976 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42910177900000002, 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093928829999999994, 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1976 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103762+0000", 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103767+0000", 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3550000000000004e-06 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103777+0000", 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5279999999999992e-06 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103784+0000", 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5739999999999998e-06 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106569+0000", 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002785151 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106668+0000", 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8807000000000005e-05 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107170+0000", 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050205400000000004 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109619+0000", 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024487469999999998 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113077+0000", 2024-09-13T09:26:03.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034580180000000002 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113120+0000", 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3112000000000003e-05 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113155+0000", 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5231999999999997e-05 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1977 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42849394400000002, 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088773199999999993, 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1977 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104369+0000", 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104390+0000", 2024-09-13T09:26:03.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0999999999999999e-05 2024-09-13T09:26:03.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104412+0000", 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2248000000000001e-05 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104423+0000", 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0321e-05 2024-09-13T09:26:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107700+0000", 2024-09-13T09:26:03.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032770329999999999 2024-09-13T09:26:03.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107828+0000", 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012850399999999999 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.108517+0000", 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068841499999999995 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111988+0000", 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034716550000000001 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113166+0000", 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001177893 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113215+0000", 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8758999999999999e-05 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113248+0000", 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2670999999999999e-05 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1979 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42599187100000002, 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085138350000000008, 2024-09-13T09:26:03.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1979 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106871+0000", 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106880+0000", 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0760000000000003e-06 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106902+0000", 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2014e-05 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106911+0000", 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7410000000000004e-06 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110772+0000", 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038607749999999999 2024-09-13T09:26:03.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110883+0000", 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011127400000000001 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111390+0000", 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050661 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115162+0000", 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037724690000000001 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115242+0000", 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0073000000000004e-05 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115303+0000", 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0631000000000003e-05 2024-09-13T09:26:03.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115386+0000", 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3181000000000002e-05 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1981 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42360387599999999, 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083091610000000007, 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1981 2024-09-13T09:26:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109259+0000", 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109267+0000", 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109280+0000", 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2862e-05 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109288+0000", 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7409999999999998e-06 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113375+0000", 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040869579999999999 2024-09-13T09:26:03.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113466+0000", 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0756999999999999e-05 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113913+0000", 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00044651000000000001 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115567+0000", 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016543949999999999 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117501+0000", 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001933702 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117544+0000", 2024-09-13T09:26:03.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3624000000000001e-05 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117570+0000", 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5347e-05 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1987 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.112985+0000", 2024-09-13T09:26:03.581 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.41987914199999998, 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792029999999995, 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1987 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112985+0000", 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112985+0000", 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112985+0000", 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112989+0000", 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4830000000000001e-06 2024-09-13T09:26:03.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112996+0000", 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5230000000000001e-06 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112999+0000", 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8399999999999997e-06 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117756+0000", 2024-09-13T09:26:03.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047566179999999998 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117827+0000", 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0710000000000006e-05 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.118142+0000", 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000315333 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.118914+0000", 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077208099999999998 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.121108+0000", 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021936009999999999 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.121140+0000", 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2446000000000001e-05 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.121164+0000", 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4190999999999999e-05 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2112 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1015808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.214237+0000", 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.31862691300000001, 2024-09-13T09:26:03.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079620969999999996, 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2112 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.214237+0000", 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.214237+0000", 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.214237+0000", 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.214241+0000", 2024-09-13T09:26:03.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.215850+0000", 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001608567 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.215887+0000", 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7042000000000002e-05 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.221949+0000", 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060620159999999999 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.222071+0000", 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121614 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.222200+0000", 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012888600000000001 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2126 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 163840~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216142+0000", 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.31672270699999999, 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007972504, 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2126 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216142+0000", 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216142+0000", 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216141+0000", 2024-09-13T09:26:03.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216156+0000", 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5424999999999999e-05 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216165+0000", 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.303e-06 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216171+0000", 2024-09-13T09:26:03.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7259999999999996e-06 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.223831+0000", 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0076597189999999997 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.223957+0000", 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012587399999999999 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224114+0000", 2024-09-13T09:26:03.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015708399999999999 2024-09-13T09:26:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.600 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2127 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.316687092, 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082727870000000002, 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.601 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2127 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216180+0000", 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.399e-06 2024-09-13T09:26:03.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216188+0000", 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9699999999999999e-06 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216195+0000", 2024-09-13T09:26:03.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.725e-06 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224153+0000", 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079579420000000008 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224278+0000", 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012521799999999999 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224450+0000", 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000172032 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.607 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2128 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.608 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:03.608 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.31666363199999997, 2024-09-13T09:26:03.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085956320000000006, 2024-09-13T09:26:03.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:03.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2128 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216200+0000", 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216204+0000", 2024-09-13T09:26:03.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6399999999999999e-06 2024-09-13T09:26:03.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216212+0000", 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1270000000000003e-06 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216219+0000", 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7399999999999998e-06 2024-09-13T09:26:03.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224493+0000", 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082740169999999998 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224636+0000", 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014311700000000001 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224796+0000", 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016049 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2129 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.31663954700000002, 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090836349999999996, 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2129 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216224+0000", 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216228+0000", 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.428e-06 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216235+0000", 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216242+0000", 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.973e-06 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224846+0000", 2024-09-13T09:26:03.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086038030000000001 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225053+0000", 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020684800000000001 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225308+0000", 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025549900000000001 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2135 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.225386+0000", 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30747836299999998, 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079289410000000001, 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2135 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.620 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225386+0000", 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225386+0000", 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225385+0000", 2024-09-13T09:26:03.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225389+0000", 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1020000000000001e-06 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225393+0000", 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5120000000000002e-06 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225397+0000", 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.822e-06 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.228644+0000", 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032475329999999999 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.228711+0000", 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6100999999999996e-05 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.228973+0000", 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026207800000000001 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232318+0000", 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033451119999999999 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233212+0000", 2024-09-13T09:26:03.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00089395299999999996 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233275+0000", 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.368e-05 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233315+0000", 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9552999999999999e-05 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.625 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2136 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30648731600000001, 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086734050000000004, 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2136 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226376+0000", 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226386+0000", 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0789e-05 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226416+0000", 2024-09-13T09:26:03.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9660000000000001e-05 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226437+0000", 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0982999999999999e-05 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.229652+0000", 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032148680000000001 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.229719+0000", 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7087000000000003e-05 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.230441+0000", 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072155899999999998 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232323+0000", 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018823069999999999 2024-09-13T09:26:03.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234949+0000", 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002625966 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235010+0000", 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1101999999999999e-05 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235050+0000", 2024-09-13T09:26:03.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0454999999999998e-05 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2137 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30641818900000001, 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095282279999999997, 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2137 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:03.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226450+0000", 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226463+0000", 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3338e-05 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226467+0000", 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.481e-06 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231090+0000", 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046230450000000001 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231197+0000", 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106604 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231659+0000", 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000462189 2024-09-13T09:26:03.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234788+0000", 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031288269999999998 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235890+0000", 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001101929 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235929+0000", 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8714000000000002e-05 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235974+0000", 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5611000000000001e-05 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2138 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30639341799999997, 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012214620000000001, 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2138 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:03.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226474+0000", 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:26:03.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226485+0000", 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226501+0000", 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6671999999999999e-05 2024-09-13T09:26:03.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232383+0000", 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058812739999999997 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232455+0000", 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2107999999999995e-05 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232795+0000", 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000340532 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234797+0000", 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020017889999999999 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238578+0000", 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003780533 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238644+0000", 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5840999999999995e-05 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238686+0000", 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1984000000000003e-05 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2139 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.30635324000000003, 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012363097999999999, 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2139 2024-09-13T09:26:03.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226514+0000", 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.608e-06 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226519+0000", 2024-09-13T09:26:03.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226523+0000", 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9940000000000002e-06 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233419+0000", 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006896297 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233548+0000", 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012899999999999999 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234129+0000", 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058001899999999998 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237501+0000", 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033728790000000001 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238760+0000", 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001258388 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238831+0000", 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0967e-05 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238874+0000", 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3439999999999997e-05 2024-09-13T09:26:03.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.068141473999999994, 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:03.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:03.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:03.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:03.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:03.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:03.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:03.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:03.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:03.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:03.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:03.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:03.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:03.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:03.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.663 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.043799211999999997, 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:03.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:03.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:03.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:03.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:03.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:03.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:03.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:03.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:03.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:03.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:03.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:03.675 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.039430848999999997, 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:03.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:03.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:03.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:03.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:03.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:03.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:03.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:03.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:03.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:03.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:03.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:03.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:03.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:03.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:03.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:03.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:03.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:03.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:03.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:03.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:03.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:03.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:03.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:03.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:03.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:03.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:03.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:03.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:03.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:03.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:03.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:03.689 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:03.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:03.693 INFO:teuthology.orchestra.run.smithi063.stdout:310 write 0x584b857 thru 0x584d254 (0x19fe bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:312 write 0xa0bbec8 thru 0xa0cbc55 (0xfd8e bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:314 trunc from 0xeda7706 to 0x99e2ca7 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:317 read 0x4d7e84e thru 0x4d8cb34 (0xe2e7 bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:318 write 0x80ba20c thru 0x80c3b2a (0x991f bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:320 punch from 0x915a29f to 0x91617f3, (0x7554 bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:321 trunc from 0x99e2ca7 to 0xd7f31a4 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:322 write 0xaee2e99 thru 0xaee5289 (0x23f1 bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:324 write 0xe490745 thru 0xe4938b0 (0x316c bytes) 2024-09-13T09:26:03.694 INFO:teuthology.orchestra.run.smithi063.stdout:325 trunc from 0xe4938b1 to 0x4536a86 2024-09-13T09:26:03.695 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:03.686+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:03.695 INFO:teuthology.orchestra.run.smithi063.stdout:326 writesame 0x4edb33f thru 0x4edd6d2 data_size 0x9(0x2394 bytes) 2024-09-13T09:26:03.696 INFO:teuthology.orchestra.run.smithi063.stdout:328 punch from 0x2af64db to 0x2afdcc5, (0x77ea bytes) 2024-09-13T09:26:03.696 INFO:teuthology.orchestra.run.smithi063.stdout:330 read 0x19eabd6 thru 0x19f22bf (0x76ea bytes) 2024-09-13T09:26:03.696 INFO:teuthology.orchestra.run.smithi063.stdout:331 writesame 0x14cc893 thru 0x14d9ba6 data_size 0xab(0xd314 bytes) 2024-09-13T09:26:03.701 INFO:teuthology.orchestra.run.smithi063.stdout:332 write 0x2d5b185 thru 0x2d683d7 (0xd253 bytes) 2024-09-13T09:26:03.710 INFO:teuthology.orchestra.run.smithi063.stdout:333 write 0x4eba3f3 thru 0x4ec802e (0xdc3c bytes) 2024-09-13T09:26:03.721 INFO:teuthology.orchestra.run.smithi063.stdout:335 write 0x1a18b16 thru 0x1a1d762 (0x4c4d bytes) 2024-09-13T09:26:03.726 INFO:teuthology.orchestra.run.smithi063.stdout:336 read 0x7bfd1c thru 0x7c1565 (0x184a bytes) 2024-09-13T09:26:03.726 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.726 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:03.726 INFO:teuthology.orchestra.run.smithi063.stdout:339 punch from 0x285218c to 0x2853064, (0xed8 bytes) 2024-09-13T09:26:03.726 INFO:teuthology.orchestra.run.smithi063.stdout:341 write 0x2f6ab31 thru 0x2f75264 (0xa734 bytes) 2024-09-13T09:26:03.747 INFO:teuthology.orchestra.run.smithi063.stdout:342 trunc from 0x4edd6d3 to 0xadaaf5b 2024-09-13T09:26:03.765 INFO:teuthology.orchestra.run.smithi063.stdout:345 trunc from 0xadaaf5b to 0xd2cc621 2024-09-13T09:26:03.775 INFO:teuthology.orchestra.run.smithi063.stdout:346 read 0x9001022 thru 0x9003458 (0x2437 bytes) 2024-09-13T09:26:03.775 INFO:teuthology.orchestra.run.smithi063.stdout:347 punch from 0x7d64551 to 0x7d68972, (0x4421 bytes) 2024-09-13T09:26:03.775 INFO:teuthology.orchestra.run.smithi063.stdout:349 write 0xae9fb10 thru 0xaea6668 (0x6b59 bytes) 2024-09-13T09:26:03.783 INFO:teuthology.orchestra.run.smithi063.stdout:350 punch from 0x15bf036 to 0x15bf0a7, (0x71 bytes) 2024-09-13T09:26:03.783 INFO:teuthology.orchestra.run.smithi063.stdout:352 read 0x7dbf859 thru 0x7dcbd3e (0xc4e6 bytes) 2024-09-13T09:26:03.783 INFO:teuthology.orchestra.run.smithi063.stdout:353 clone 4 order 24 su 1048576 sc 6 2024-09-13T09:26:03.792 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:03.786+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:03.809 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:03.810 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:03.810 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:03.810 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:03.819 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:03.892 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:03.885+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:03.924 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:03.924 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:03.924 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:03.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:03.993 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:03.986+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:04.045 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:04.045 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:04.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:04.045 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:04.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:199 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7269387250000001, 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060758599999999998, 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 199 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:04.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313896+0000", 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313906+0000", 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1870000000000003e-06 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313920+0000", 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3871e-05 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313940+0000", 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0582999999999999e-05 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.314070+0000", 2024-09-13T09:26:04.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013007399999999999 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.314087+0000", 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6413e-05 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317280+0000", 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003193244 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317574+0000", 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029457700000000002 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317844+0000", 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000269831 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319224+0000", 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001379665 2024-09-13T09:26:04.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319823+0000", 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059955700000000002 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319911+0000", 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319974+0000", 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2651999999999996e-05 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:260 3.a 3:55f6faff:::rbd_data.111867511a81.00000000000000eb:head [set-alloc-hint object_size 1048576 write_size 1048576,write 600819~13703] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5833252709999996, 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002930972, 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 260 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457509+0000", 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457561+0000", 2024-09-13T09:26:04.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1952e-05 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457595+0000", 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4269e-05 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457620+0000", 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.459e-05 2024-09-13T09:26:04.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457750+0000", 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013036699999999999 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458002+0000", 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025142099999999999 2024-09-13T09:26:04.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458263+0000", 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026117700000000002 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.459691+0000", 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014285159999999999 2024-09-13T09:26:04.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460298+0000", 2024-09-13T09:26:04.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00060671200000000003 2024-09-13T09:26:04.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460383+0000", 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4834999999999997e-05 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460443+0000", 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9616e-05 2024-09-13T09:26:04.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4817066260000002, 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:04.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:297 3.10 e27/25 ondisk, result = 0)", 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4398497030000001, 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006064926, 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600986+0000", 2024-09-13T09:26:04.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600992+0000", 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7699999999999998e-06 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.601009+0000", 2024-09-13T09:26:04.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7197000000000002e-05 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.601020+0000", 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0648999999999999e-05 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606853+0000", 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058327409999999998 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606867+0000", 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.43e-05 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607052+0000", 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018516600000000001 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:301 3.a 3:51773985:::rbd_data.111867511a81.00000000000000ca:head [set-alloc-hint object_size 1048576 write_size 1048576,write 132677~64267] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.617290+0000", 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4235474080000001, 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002226352, 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 301 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617290+0000", 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617290+0000", 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617288+0000", 2024-09-13T09:26:04.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617430+0000", 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000141743 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617443+0000", 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2996000000000001e-05 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617449+0000", 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.144e-06 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617550+0000", 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100961 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617793+0000", 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000243181 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.617991+0000", 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019767600000000001 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619410+0000", 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001419093 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619450+0000", 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0034999999999999e-05 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619489+0000", 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8488999999999997e-05 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.619516+0000", 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7331e-05 2024-09-13T09:26:04.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:324 3.a 3:51773985:::rbd_data.111867511a81.00000000000000ca:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3637759620000001, 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003713062, 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 324 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:04.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677059+0000", 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677071+0000", 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2133000000000001e-05 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677095+0000", 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4015e-05 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677106+0000", 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0542e-05 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677216+0000", 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110306 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677397+0000", 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181056 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677562+0000", 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016472500000000001 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678899+0000", 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013370470000000001 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680639+0000", 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001739864 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680712+0000", 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3681000000000001e-05 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680774+0000", 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865000000000005e-05 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:347 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:04.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3283956220000004, 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045695639999999999, 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 347 2024-09-13T09:26:04.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712439+0000", 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712454+0000", 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5228000000000001e-05 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712490+0000", 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5577000000000002e-05 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712517+0000", 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7582000000000001e-05 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712595+0000", 2024-09-13T09:26:04.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8011999999999998e-05 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712678+0000", 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2559999999999996e-05 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712778+0000", 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9698000000000007e-05 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714327+0000", 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001549784 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716842+0000", 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025141489999999998 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716940+0000", 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8374000000000004e-05 2024-09-13T09:26:04.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717011+0000", 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1023000000000001e-05 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.092 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:518 3.1e e27/21 3:7c1f840a:::rbd_data.111867511a81.00000000000000d0:head v 27'7, mlcod=27'5)", 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.9041888409999999, 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031587289999999999, 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136646+0000", 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136823+0000", 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017679600000000001 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136851+0000", 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8195e-05 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136870+0000", 2024-09-13T09:26:04.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.931e-05 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136992+0000", 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121146 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.137050+0000", 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8142e-05 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.139718+0000", 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026680380000000002 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.139807+0000", 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9165000000000004e-05 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.097 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:565 3.10 3:0e125993:::rbd_data.111867511a81.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write 75546~42715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.245179+0000", 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7956574779999999, 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026099600000000001, 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 565 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245179+0000", 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245179+0000", 2024-09-13T09:26:04.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245176+0000", 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245258+0000", 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1635e-05 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245303+0000", 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5448999999999998e-05 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245328+0000", 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4363e-05 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245415+0000", 2024-09-13T09:26:04.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7107999999999993e-05 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245541+0000", 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000126549 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.245672+0000", 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000130383 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.246845+0000", 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011733469999999999 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.247675+0000", 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00082975699999999998 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.247744+0000", 2024-09-13T09:26:04.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8924000000000006e-05 2024-09-13T09:26:04.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.247789+0000", 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5785000000000003e-05 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:1862 3.1e e30/21 3:7913b972:::rbd_data.1118e3e5e25d.00000000000000c7:head v 30'13, mlcod=30'11)", 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.087793+0000", 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.95304410699999997, 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003437973, 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087793+0000", 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087793+0000", 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087791+0000", 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087805+0000", 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3902e-05 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087834+0000", 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8345000000000001e-05 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087850+0000", 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6512e-05 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087937+0000", 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6398999999999997e-05 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087993+0000", 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6820999999999997e-05 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091183+0000", 2024-09-13T09:26:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031895890000000001 2024-09-13T09:26:04.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091231+0000", 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7895000000000001e-05 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:1876 3.1e e30/21 3:7d1bc93e:::rbd_data.1118e3e5e25d.000000000000000e:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100184+0000", 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.94065334599999995, 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003290608, 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100184+0000", 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100184+0000", 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100183+0000", 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100194+0000", 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1018e-05 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100210+0000", 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6018000000000001e-05 2024-09-13T09:26:04.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100218+0000", 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100279+0000", 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1793999999999998e-05 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100316+0000", 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6248000000000003e-05 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103430+0000", 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003114688 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103474+0000", 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3841000000000003e-05 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.153 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2142 3.10 3:09299c6d:::rbd_data.1118e3e5e25d.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 86795] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.232591+0000", 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.80824598400000003, 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002276082, 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.154 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2142 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232591+0000", 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232591+0000", 2024-09-13T09:26:04.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232588+0000", 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232605+0000", 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6489000000000001e-05 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232633+0000", 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7988999999999999e-05 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232646+0000", 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3086e-05 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232724+0000", 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7541999999999997e-05 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232949+0000", 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000225675 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233215+0000", 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026571100000000002 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234070+0000", 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00085517500000000001 2024-09-13T09:26:04.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234782+0000", 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00071144899999999996 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234818+0000", 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6029999999999999e-05 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234867+0000", 2024-09-13T09:26:04.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9418e-05 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:2143 3.1e e30/21 3:7a2e114f:::rbd_data.1118e3e5e25d.0000000000000009:head v 30'18, mlcod=30'16)", 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.234310+0000", 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.80652666799999995, 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030434139999999999, 2024-09-13T09:26:04.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234310+0000", 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234310+0000", 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234309+0000", 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234318+0000", 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9420000000000001e-06 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234335+0000", 2024-09-13T09:26:04.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6657e-05 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234350+0000", 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5146e-05 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234419+0000", 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9595e-05 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234441+0000", 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1871000000000001e-05 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237317+0000", 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028752579999999999 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237354+0000", 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7082000000000002e-05 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2218 3.10 3:0a24612b:::rbd_data.1118e3e5e25d.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.371303+0000", 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.66953430899999999, 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039573680000000002, 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2218 2024-09-13T09:26:04.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371303+0000", 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371303+0000", 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371300+0000", 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371316+0000", 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371355+0000", 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9035999999999997e-05 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371375+0000", 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9811000000000001e-05 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371448+0000", 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2527999999999994e-05 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371527+0000", 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9525999999999997e-05 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371625+0000", 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7709999999999998e-05 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.373856+0000", 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022314370000000002 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.375176+0000", 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013200659999999999 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.375221+0000", 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4209999999999999e-05 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.375260+0000", 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9338000000000002e-05 2024-09-13T09:26:04.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2243 3.a 3:55dca6b4:::rbd_data.1118e3e5e25d.00000000000000be:head [set-alloc-hint object_size 262144 write_size 262144,write 170058~26550] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.422361+0000", 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.61847637600000005, 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026644899999999998, 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2243 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422361+0000", 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422361+0000", 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422358+0000", 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422395+0000", 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7601999999999997e-05 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422428+0000", 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2487999999999998e-05 2024-09-13T09:26:04.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422442+0000", 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4645e-05 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422552+0000", 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010949499999999999 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.422833+0000", 2024-09-13T09:26:04.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000281544 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.423138+0000", 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030504300000000001 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.424737+0000", 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015990119999999999 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.424972+0000", 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234625 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.425002+0000", 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0287999999999999e-05 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.425025+0000", 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2790999999999999e-05 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.176 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2255 3.10 3:08708bba:::rbd_data.1118e3e5e25d.000000000000014f:head [set-alloc-hint object_size 262144 write_size 262144,write 134158~6524] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.457614+0000", 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58322279099999996, 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027052809999999999, 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2255 2024-09-13T09:26:04.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457614+0000", 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457614+0000", 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457612+0000", 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457633+0000", 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1234e-05 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457665+0000", 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1758999999999999e-05 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457688+0000", 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3042999999999999e-05 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457771+0000", 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2675000000000004e-05 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.457956+0000", 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018580100000000001 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.458126+0000", 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016931800000000001 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.459336+0000", 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012106370000000001 2024-09-13T09:26:04.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.460189+0000", 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00085282499999999998 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.460272+0000", 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2713000000000006e-05 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.460319+0000", 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7605000000000002e-05 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2290 3.a 3:5333385b:::rbd_data.1118e3e5e25d.000000000000027d:head [set-alloc-hint object_size 262144 write_size 262144,write 229376~15446] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.549143+0000", 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.49169380400000001, 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025772999999999998, 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2290 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549143+0000", 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549143+0000", 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549141+0000", 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549186+0000", 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5364000000000001e-05 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549232+0000", 2024-09-13T09:26:04.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5784000000000001e-05 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549254+0000", 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2470999999999999e-05 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549335+0000", 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0558999999999999e-05 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549470+0000", 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013478700000000001 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.549657+0000", 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018764399999999999 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.550548+0000", 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00089052000000000001 2024-09-13T09:26:04.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.551601+0000", 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001053582 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.551675+0000", 2024-09-13T09:26:04.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3565000000000005e-05 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.551720+0000", 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5429000000000002e-05 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2294 3.a 3:5333385b:::rbd_data.1118e3e5e25d.000000000000027d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.565337+0000", 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.47549987100000002, 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021981990000000001, 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2294 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565337+0000", 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565337+0000", 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565336+0000", 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565343+0000", 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6669999999999999e-06 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565358+0000", 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5696e-05 2024-09-13T09:26:04.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565366+0000", 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565500+0000", 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013353400000000001 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565719+0000", 2024-09-13T09:26:04.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021939799999999999 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.565993+0000", 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027332799999999998 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.567098+0000", 2024-09-13T09:26:04.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011052340000000001 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.567379+0000", 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000280835 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.567455+0000", 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6137999999999995e-05 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.567535+0000", 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0329999999999998e-05 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2332 3.10 3:08708bba:::rbd_data.1118e3e5e25d.000000000000014f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.652283+0000", 2024-09-13T09:26:04.196 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.38855424300000002, 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061894089999999999, 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2332 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652283+0000", 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652283+0000", 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652280+0000", 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652296+0000", 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5418999999999999e-05 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652326+0000", 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0602999999999998e-05 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652342+0000", 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5486e-05 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652552+0000", 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209846 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652822+0000", 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027073700000000001 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.653090+0000", 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000267724 2024-09-13T09:26:04.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.654470+0000", 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001379786 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.658307+0000", 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003836711 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.658404+0000", 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6921000000000003e-05 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.658472+0000", 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8582000000000002e-05 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2387 3.10 3:0842da77:::rbd_data.1118e3e5e25d.00000000000000bb:head [set-alloc-hint object_size 262144 write_size 262144,write 142129~21711] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.729603+0000", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.31123442600000001, 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002420457, 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2387 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729603+0000", 2024-09-13T09:26:04.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729603+0000", 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729600+0000", 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729646+0000", 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5488999999999998e-05 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729672+0000", 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6491000000000001e-05 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729687+0000", 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.429e-05 2024-09-13T09:26:04.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.729803+0000", 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115835 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.730084+0000", 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028171799999999997 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.730296+0000", 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021215900000000001 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.731537+0000", 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00124091 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.731924+0000", 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038618399999999997 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.731986+0000", 2024-09-13T09:26:04.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2051999999999995e-05 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.732023+0000", 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7438999999999998e-05 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.208 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:04.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:04.210 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:04.202+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:04.270 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:26:04.311 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:04.305+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:04.337 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:04.337 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:04.337 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:04.337 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:04.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:04.413 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:04.405+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:04.420 INFO:teuthology.orchestra.run.smithi063.stdout:flattening image image_client.0-clone3 2024-09-13T09:26:04.452 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:04.453 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:04.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:04.514 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:04.506+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:04.572 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.605526803, 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:04.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:04.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.609137348000001, 2024-09-13T09:26:04.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:04.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:04.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.201034829000001, 2024-09-13T09:26:04.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0121125, 2024-09-13T09:26:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 108 2024-09-13T09:26:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:26:04.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:26:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365505+0000", 2024-09-13T09:26:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365518+0000", 2024-09-13T09:26:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2456000000000001e-05 2024-09-13T09:26:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365553+0000", 2024-09-13T09:26:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5086000000000003e-05 2024-09-13T09:26:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365567+0000", 2024-09-13T09:26:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4187e-05 2024-09-13T09:26:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371405+0000", 2024-09-13T09:26:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058378960000000004 2024-09-13T09:26:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371551+0000", 2024-09-13T09:26:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014597300000000001 2024-09-13T09:26:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371737+0000", 2024-09-13T09:26:04.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018655899999999999 2024-09-13T09:26:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377392+0000", 2024-09-13T09:26:04.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056547380000000003 2024-09-13T09:26:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377484+0000", 2024-09-13T09:26:04.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2261000000000003e-05 2024-09-13T09:26:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377565+0000", 2024-09-13T09:26:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0464999999999996e-05 2024-09-13T09:26:04.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377619+0000", 2024-09-13T09:26:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4221999999999997e-05 2024-09-13T09:26:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.604 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.200563716000001, 2024-09-13T09:26:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014920113, 2024-09-13T09:26:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-13T09:26:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365977+0000", 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365987+0000", 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366004+0000", 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.662e-05 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366022+0000", 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7883000000000001e-05 2024-09-13T09:26:04.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376091+0000", 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010069171999999999 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376246+0000", 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000155377 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376548+0000", 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030137300000000002 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380798+0000", 2024-09-13T09:26:04.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004250406 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380813+0000", 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4357e-05 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380862+0000", 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9137000000000001e-05 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380898+0000", 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6158999999999997e-05 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.200505138, 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016811104, 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366035+0000", 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366045+0000", 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0327e-05 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366062+0000", 2024-09-13T09:26:04.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6483000000000001e-05 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366074+0000", 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377874+0000", 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011799983999999999 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378006+0000", 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013130100000000001 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378143+0000", 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137157 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380914+0000", 2024-09-13T09:26:04.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002771281 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382760+0000", 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001846253 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382810+0000", 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9483e-05 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382847+0000", 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7749000000000001e-05 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.200425436, 2024-09-13T09:26:04.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018719919000000002, 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:04.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366115+0000", 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366313+0000", 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000198156 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366331+0000", 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7739000000000001e-05 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366345+0000", 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4226000000000001e-05 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378684+0000", 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012339325 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378827+0000", 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014284700000000001 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.379116+0000", 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028930399999999999 2024-09-13T09:26:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382675+0000", 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035590050000000001 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384568+0000", 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001892707 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384706+0000", 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013804099999999999 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384836+0000", 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012977900000000001 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.200179278, 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01866932, 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:04.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366361+0000", 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366372+0000", 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0857e-05 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366388+0000", 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366401+0000", 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.326e-05 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380005+0000", 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013604036 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380100+0000", 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4468000000000004e-05 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380211+0000", 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011084199999999999 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383662+0000", 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034510460000000001 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384905+0000", 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001243009 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384996+0000", 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:26:04.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385032+0000", 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5528000000000003e-05 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.200126429000001, 2024-09-13T09:26:04.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020964449999999999, 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366414+0000", 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366423+0000", 2024-09-13T09:26:04.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366443+0000", 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366455+0000", 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380966+0000", 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014510662000000001 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381062+0000", 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6526000000000004e-05 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381259+0000", 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019658499999999999 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383675+0000", 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024160480000000001 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387295+0000", 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036198020000000001 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387342+0000", 2024-09-13T09:26:04.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7299000000000002e-05 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387380+0000", 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7379000000000002e-05 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:04.638 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.199702402, 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020678247, 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366838+0000", 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367071+0000", 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023322800000000001 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367094+0000", 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3079999999999999e-05 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367111+0000", 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7008999999999999e-05 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381788+0000", 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014676850999999999 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381894+0000", 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010610799999999999 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382045+0000", 2024-09-13T09:26:04.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015073699999999999 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383689+0000", 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016436630000000001 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387445+0000", 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037564 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387488+0000", 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3402000000000002e-05 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387517+0000", 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8975999999999998e-05 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.199412715000001, 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020557408999999999, 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.645 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367128+0000", 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367138+0000", 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0672999999999999e-05 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367157+0000", 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8779000000000001e-05 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367174+0000", 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6464e-05 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382926+0000", 2024-09-13T09:26:04.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015752759000000002 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383077+0000", 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150965 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383228+0000", 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015101399999999999 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384458+0000", 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012294370000000001 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387599+0000", 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031411960000000002 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387647+0000", 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7648999999999997e-05 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387686+0000", 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9653000000000001e-05 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.199353609999999, 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022933500999999998, 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-13T09:26:04.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367187+0000", 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367208+0000", 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1793e-05 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367225+0000", 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668999999999999e-05 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367239+0000", 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4005999999999999e-05 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383748+0000", 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016508719000000002 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383933+0000", 2024-09-13T09:26:04.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018481500000000001 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384122+0000", 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189626 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385049+0000", 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092678599999999997 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390003+0000", 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049541819999999997 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390075+0000", 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2094999999999993e-05 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390121+0000", 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6082999999999999e-05 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.609464601999999, 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012255703999999999, 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 134 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:26:04.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957076+0000", 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957179+0000", 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103136 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957213+0000", 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4422999999999999e-05 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957225+0000", 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1178e-05 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963332+0000", 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061077019999999996 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963543+0000", 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021082699999999999 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963933+0000", 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038984300000000001 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967237+0000", 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033042449999999999 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969266+0000", 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020291200000000001 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969303+0000", 2024-09-13T09:26:04.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6183000000000002e-05 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969333+0000", 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0063e-05 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:04.664 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.60929823, 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015204896000000001, 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 135 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957242+0000", 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957250+0000", 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957268+0000", 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7459999999999999e-05 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957277+0000", 2024-09-13T09:26:04.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3559999999999999e-06 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965456+0000", 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792359999999995 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965567+0000", 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110877 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965741+0000", 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017349600000000001 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967304+0000", 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001563444 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972380+0000", 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050761019999999999 2024-09-13T09:26:04.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972425+0000", 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4366000000000002e-05 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972448+0000", 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3455000000000002e-05 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.609254159999999, 2024-09-13T09:26:04.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015208101, 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 136 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957297+0000", 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9909999999999997e-06 2024-09-13T09:26:04.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957339+0000", 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2267e-05 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957371+0000", 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2518000000000003e-05 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966255+0000", 2024-09-13T09:26:04.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088832679999999997 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966353+0000", 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8658000000000003e-05 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966486+0000", 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013309500000000001 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969257+0000", 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027707209999999999 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972453+0000", 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031962789999999998 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972478+0000", 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4655e-05 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972495+0000", 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7407000000000001e-05 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.609139298999999, 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015156871000000001, 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957401+0000", 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957509+0000", 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010808 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957536+0000", 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6237e-05 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957542+0000", 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2020000000000001e-06 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967352+0000", 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098099580000000006 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967447+0000", 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5375000000000001e-05 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967584+0000", 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136463 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971541+0000", 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039574930000000003 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972499+0000", 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095777799999999995 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972527+0000", 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7900999999999999e-05 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972559+0000", 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2191999999999999e-05 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.608989462, 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015104328, 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957551+0000", 2024-09-13T09:26:04.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957556+0000", 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.318e-06 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957565+0000", 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.115e-06 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957572+0000", 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968300+0000", 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010727617 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968387+0000", 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7021000000000006e-05 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968482+0000", 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5081999999999995e-05 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971595+0000", 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031126639999999998 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972588+0000", 2024-09-13T09:26:04.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099296600000000007 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972635+0000", 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7136999999999999e-05 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972656+0000", 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1480000000000001e-05 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.608960661000001, 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015429492, 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-13T09:26:04.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957580+0000", 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957586+0000", 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957620+0000", 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3215e-05 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957651+0000", 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1027e-05 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969570+0000", 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011919511000000001 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969674+0000", 2024-09-13T09:26:04.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104248 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969784+0000", 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010937700000000001 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972662+0000", 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028781589999999999 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972925+0000", 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026289899999999998 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972976+0000", 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0702999999999999e-05 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973010+0000", 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4743000000000003e-05 2024-09-13T09:26:04.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.608856057000001, 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020098000000000001, 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957793+0000", 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108837 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957808+0000", 2024-09-13T09:26:04.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957814+0000", 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970251+0000", 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012436523999999999 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970350+0000", 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9600999999999997e-05 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970506+0000", 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015539500000000001 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972667+0000", 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002161619 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977681+0000", 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050135930000000002 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977722+0000", 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1634999999999997e-05 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977783+0000", 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0968000000000003e-05 2024-09-13T09:26:04.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.608719929999999, 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022700502000000001, 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957821+0000", 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957825+0000", 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957843+0000", 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7492000000000001e-05 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957851+0000", 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971666+0000", 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013814707000000001 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971841+0000", 2024-09-13T09:26:04.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174659 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971976+0000", 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013485600000000001 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973161+0000", 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011852130000000001 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980427+0000", 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072666440000000001 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980487+0000", 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9373000000000002e-05 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980522+0000", 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5244999999999999e-05 2024-09-13T09:26:04.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.790852002000001, 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013540165999999999, 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:04.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775694+0000", 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8520000000000003e-06 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775699+0000", 2024-09-13T09:26:04.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.682e-06 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775709+0000", 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3980000000000005e-06 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783188+0000", 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074792560000000001 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:04.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783392+0000", 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020345199999999999 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783653+0000", 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026178699999999998 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.785173+0000", 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001519739 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:04.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789105+0000", 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039318900000000004 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789174+0000", 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9568999999999997e-05 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789230+0000", 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5169e-05 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:04.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:04.713 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:04.714 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T09:26:04.714 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:04.709+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:04.715 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:04.781 INFO:teuthology.orchestra.run.smithi063.stdout:355 trunc from 0xd2cc621 to 0xe25cf7a 2024-09-13T09:26:04.796 INFO:teuthology.orchestra.run.smithi063.stdout:356 read 0xf4b77b thru 0xf58e72 (0xd6f8 bytes) 2024-09-13T09:26:04.797 INFO:teuthology.orchestra.run.smithi063.stdout:358 write 0x2c230e7 thru 0x2c25b88 (0x2aa2 bytes) 2024-09-13T09:26:04.815 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:04.809+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:04.831 INFO:teuthology.orchestra.run.smithi063.stdout:359 trunc from 0xe25cf7a to 0x1c16beb 2024-09-13T09:26:04.845 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:04.845 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:04.845 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:04.845 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:04.856 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:04.856 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:04.880 INFO:teuthology.orchestra.run.smithi063.stdout:363 write 0xb87a565 thru 0xb888edb (0xe977 bytes) 2024-09-13T09:26:04.886 INFO:teuthology.orchestra.run.smithi063.stdout:365 writesame 0x210e658 thru 0x2117178 data_size 0x1(0x8b21 bytes) 2024-09-13T09:26:04.912 INFO:teuthology.orchestra.run.smithi063.stdout:366 punch from 0xaa92ffd to 0xaa9fb69, (0xcb6c bytes) 2024-09-13T09:26:04.913 INFO:teuthology.orchestra.run.smithi063.stdout:367 read 0x15652f2 thru 0x1572607 (0xd316 bytes) 2024-09-13T09:26:04.913 INFO:teuthology.orchestra.run.smithi063.stdout:368 write 0x7d37419 thru 0x7d4660e (0xf1f6 bytes) 2024-09-13T09:26:04.915 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:04.910+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:04.927 INFO:teuthology.orchestra.run.smithi063.stdout:370 writesame 0xc377f33 thru 0xc37f76f data_size 0x1(0x783d bytes) 2024-09-13T09:26:04.927 INFO:teuthology.orchestra.run.smithi063.stdout:371 punch from 0x33d9053 to 0x33ddb74, (0x4b21 bytes) 2024-09-13T09:26:04.927 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:04.927 INFO:teuthology.orchestra.run.smithi063.stdout:373 write 0x8fe8ec6 thru 0x8ff321f (0xa35a bytes) 2024-09-13T09:26:04.929 INFO:teuthology.orchestra.run.smithi063.stdout:374 punch from 0x559d177 to 0x55aa629, (0xd4b2 bytes) 2024-09-13T09:26:04.929 INFO:teuthology.orchestra.run.smithi063.stdout:375 read 0xc073879 thru 0xc077016 (0x379e bytes) 2024-09-13T09:26:04.929 INFO:teuthology.orchestra.run.smithi063.stdout:376 writesame 0xaad7160 thru 0xaadc2a1 data_size 0x3(0x5142 bytes) 2024-09-13T09:26:04.941 INFO:teuthology.orchestra.run.smithi063.stdout:377 read 0x7c98d1f thru 0x7ca6b34 (0xde16 bytes) 2024-09-13T09:26:04.942 INFO:teuthology.orchestra.run.smithi063.stdout:378 punch from 0xa308728 to 0xa316d62, (0xe63a bytes) 2024-09-13T09:26:04.942 INFO:teuthology.orchestra.run.smithi063.stdout:380 read 0x9079b3e thru 0x90808a1 (0x6d64 bytes) 2024-09-13T09:26:04.942 INFO:teuthology.orchestra.run.smithi063.stdout:382 read 0x1ad2250 thru 0x1adf929 (0xd6da bytes) 2024-09-13T09:26:04.943 INFO:teuthology.orchestra.run.smithi063.stdout:383 read 0x63c9b95 thru 0x63d358e (0x99fa bytes) 2024-09-13T09:26:04.943 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:04.943 INFO:teuthology.orchestra.run.smithi063.stdout:386 read 0xb2b7bd6 thru 0xb2be29b (0x66c6 bytes) 2024-09-13T09:26:04.944 INFO:teuthology.orchestra.run.smithi063.stdout:388 write 0x856e3b thru 0x857edc (0x10a2 bytes) 2024-09-13T09:26:04.947 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:04.963 INFO:teuthology.orchestra.run.smithi063.stdout:390 trunc from 0xc37f770 to 0x3537e0b 2024-09-13T09:26:04.971 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:04.971 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:04.971 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:04.971 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:04.971 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:04.982 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:05.009 INFO:teuthology.orchestra.run.smithi063.stdout:393 write 0xa917858 thru 0xa9180a8 (0x851 bytes) 2024-09-13T09:26:05.016 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:05.009+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:05.031 INFO:teuthology.orchestra.run.smithi063.stdout:394 trunc from 0xa9180a9 to 0x2217f5c 2024-09-13T09:26:05.052 INFO:teuthology.orchestra.run.smithi063.stdout:395 read 0x1dd923c thru 0x1ddcab7 (0x387c bytes) 2024-09-13T09:26:05.053 INFO:teuthology.orchestra.run.smithi063.stdout:396 read 0x1fff86d thru 0x2005477 (0x5c0b bytes) 2024-09-13T09:26:05.054 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.054 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.054 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.059 INFO:teuthology.orchestra.run.smithi063.stdout:401 trunc from 0x2217f5c to 0x4c29b37 2024-09-13T09:26:05.074 INFO:teuthology.orchestra.run.smithi063.stdout:402 trunc from 0x4c29b37 to 0x638c570 2024-09-13T09:26:05.105 INFO:teuthology.orchestra.run.smithi063.stdout:403 write 0xc090684 thru 0xc0957c0 (0x513d bytes) 2024-09-13T09:26:05.109 INFO:teuthology.orchestra.run.smithi063.stdout:405 write 0x934b960 thru 0x934d1fe (0x189f bytes) 2024-09-13T09:26:05.115 INFO:teuthology.orchestra.run.smithi063.stdout:407 write 0xa34dd9b thru 0xa34fb0a (0x1d70 bytes) 2024-09-13T09:26:05.117 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:05.109+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:05.121 INFO:teuthology.orchestra.run.smithi063.stdout:408 punch from 0xb061da1 to 0xb0660c2, (0x4321 bytes) 2024-09-13T09:26:05.122 INFO:teuthology.orchestra.run.smithi063.stdout:409 read 0x7509e09 thru 0x75107bd (0x69b5 bytes) 2024-09-13T09:26:05.122 INFO:teuthology.orchestra.run.smithi063.stdout:410 writesame 0x1893c77 thru 0x1895471 data_size 0x1(0x17fb bytes) 2024-09-13T09:26:05.130 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.139 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.184488231, 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:05.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:05.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.167930018, 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:05.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:05.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:05.151 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9331827599999993, 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:05.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.153 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:05.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:05.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:05.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.159 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9309736229999999, 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:05.160 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:05.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:05.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:05.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:05.165 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.165 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.165 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.166 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:05.166 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:05.166 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9266290979999994, 2024-09-13T09:26:05.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:05.167 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.167 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:05.167 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:05.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:05.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:05.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:05.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:05.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:05.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:05.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:05.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:05.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:05.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:05.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9100781189999996, 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.176 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.177 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:05.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:05.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:05.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:05.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:05.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:05.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:05.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:05.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.184 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.185 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:05.185 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:05.185 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9094901110000002, 2024-09-13T09:26:05.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:05.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.186 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:05.186 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:05.186 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:05.186 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:05.186 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:05.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.187 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:05.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:05.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:05.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:05.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:05.192 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9017839549999991, 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:05.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:05.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:05.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:05.198 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:05.199 INFO:teuthology.orchestra.run.smithi063.stdout:412 trunc from 0xc0957c1 to 0x683ba72 2024-09-13T09:26:05.200 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:05.201 INFO:teuthology.orchestra.run.smithi063.stdout:413 writesame 0x945356f thru 0x945390f data_size 0x1(0x3a1 bytes) 2024-09-13T09:26:05.201 INFO:teuthology.orchestra.run.smithi063.stdout:414 trunc from 0x9453910 to 0xab98afb 2024-09-13T09:26:05.201 INFO:teuthology.orchestra.run.smithi063.stdout:415 trunc from 0xab98afb to 0x8e6e688 2024-09-13T09:26:05.207 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.207 INFO:teuthology.orchestra.run.smithi063.stdout:417 write 0x5b3c1eb thru 0x5b440ae (0x7ec4 bytes) 2024-09-13T09:26:05.210 INFO:teuthology.orchestra.run.smithi063.stdout:418 write 0x4628364 thru 0x46293c2 (0x105f bytes) 2024-09-13T09:26:05.213 INFO:teuthology.orchestra.run.smithi063.stdout:419 read 0x705ba25 thru 0x705dd22 (0x22fe bytes) 2024-09-13T09:26:05.218 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:05.210+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:05.233 INFO:teuthology.orchestra.run.smithi063.stdout:422 writesame 0xd0feeaf thru 0xd0ff1f1 data_size 0x1(0x343 bytes) 2024-09-13T09:26:05.234 INFO:teuthology.orchestra.run.smithi063.stdout:423 punch from 0x4e63b3b to 0x4e6f763, (0xbc28 bytes) 2024-09-13T09:26:05.234 INFO:teuthology.orchestra.run.smithi063.stdout:424 writesame 0xa01f48d thru 0xa01f8d8 data_size 0x16(0x44c bytes) 2024-09-13T09:26:05.239 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.239 INFO:teuthology.orchestra.run.smithi063.stdout:426 write 0x6d51465 thru 0x6d54df2 (0x398e bytes) 2024-09-13T09:26:05.243 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.243 INFO:teuthology.orchestra.run.smithi063.stdout:429 read 0x5653538 thru 0x565ef3a (0xba03 bytes) 2024-09-13T09:26:05.244 INFO:teuthology.orchestra.run.smithi063.stdout:430 read 0xdf6039 thru 0xe004b5 (0xa47d bytes) 2024-09-13T09:26:05.245 INFO:teuthology.orchestra.run.smithi063.stdout:433 write 0xb1e5f73 thru 0xb1ef775 (0x9803 bytes) 2024-09-13T09:26:05.248 INFO:teuthology.orchestra.run.smithi063.stdout:436 write 0x7711483 thru 0x7711b82 (0x700 bytes) 2024-09-13T09:26:05.254 INFO:teuthology.orchestra.run.smithi063.stdout:437 write 0x340166e thru 0x3407505 (0x5e98 bytes) 2024-09-13T09:26:05.257 INFO:teuthology.orchestra.run.smithi063.stdout:438 read 0x8face98 thru 0x8fb2cec (0x5e55 bytes) 2024-09-13T09:26:05.258 INFO:teuthology.orchestra.run.smithi063.stdout:439 write 0x5341326 thru 0x534793a (0x6615 bytes) 2024-09-13T09:26:05.261 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.261 INFO:teuthology.orchestra.run.smithi063.stdout:442 read 0x95307fd thru 0x9535669 (0x4e6d bytes) 2024-09-13T09:26:05.262 INFO:teuthology.orchestra.run.smithi063.stdout:443 writesame 0x3bed7d thru 0x3c7370 data_size 0x2(0x85f4 bytes) 2024-09-13T09:26:05.274 INFO:teuthology.orchestra.run.smithi063.stdout:445 read 0x86380e2 thru 0x86457ff (0xd71e bytes) 2024-09-13T09:26:05.275 INFO:teuthology.orchestra.run.smithi063.stdout:446 read 0x1975921 thru 0x1983707 (0xdde7 bytes) 2024-09-13T09:26:05.275 INFO:teuthology.orchestra.run.smithi063.stdout:447 write 0x71aaa60 thru 0x71b0730 (0x5cd1 bytes) 2024-09-13T09:26:05.286 INFO:teuthology.orchestra.run.smithi063.stdout:449 writesame 0xd6d036e thru 0xd6d427b data_size 0x7(0x3f0e bytes) 2024-09-13T09:26:05.286 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.286 INFO:teuthology.orchestra.run.smithi063.stdout:451 punch from 0x40b50cb to 0x40ba2df, (0x5214 bytes) 2024-09-13T09:26:05.296 INFO:teuthology.orchestra.run.smithi063.stdout:452 write 0xdcb6a39 thru 0xdcc4dd5 (0xe39d bytes) 2024-09-13T09:26:05.302 INFO:teuthology.orchestra.run.smithi063.stdout:454 write 0xb6898fb thru 0xb68bbba (0x22c0 bytes) 2024-09-13T09:26:05.317 INFO:teuthology.orchestra.run.smithi063.stdout:455 trunc from 0xdcc4dd6 to 0x8470a3f 2024-09-13T09:26:05.318 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:05.310+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:05.338 INFO:teuthology.orchestra.run.smithi063.stdout:456 punch from 0x65dffe3 to 0x65e4a21, (0x4a3e bytes) 2024-09-13T09:26:05.340 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:05.340 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:05.341 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:05.341 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:05.343 INFO:teuthology.orchestra.run.smithi063.stdout:457 trunc from 0x8470a3f to 0xac12552 2024-09-13T09:26:05.351 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.351 INFO:teuthology.orchestra.run.smithi063.stdout:459 write 0x8b97fbd thru 0x8b9fea9 (0x7eed bytes) 2024-09-13T09:26:05.352 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:05.356 INFO:teuthology.orchestra.run.smithi063.stdout:462 read 0x6b8dc2b thru 0x6b956d1 (0x7aa7 bytes) 2024-09-13T09:26:05.356 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.356 INFO:teuthology.orchestra.run.smithi063.stdout:464 read 0x777d3ba thru 0x7785969 (0x85b0 bytes) 2024-09-13T09:26:05.357 INFO:teuthology.orchestra.run.smithi063.stdout:465 read 0x2b7b302 thru 0x2b849cf (0x96ce bytes) 2024-09-13T09:26:05.357 INFO:teuthology.orchestra.run.smithi063.stdout:466 write 0xabd41cd thru 0xabde0a3 (0x9ed7 bytes) 2024-09-13T09:26:05.359 INFO:teuthology.orchestra.run.smithi063.stdout:467 read 0x21973bf thru 0x21a1dba (0xa9fc bytes) 2024-09-13T09:26:05.367 INFO:teuthology.orchestra.run.smithi063.stdout:469 trunc from 0xac12552 to 0x6a86deb 2024-09-13T09:26:05.367 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:05.382 INFO:teuthology.orchestra.run.smithi063.stdout:470 read 0x2a83f16 thru 0x2a85c7d (0x1d68 bytes) 2024-09-13T09:26:05.382 INFO:teuthology.orchestra.run.smithi063.stdout:471 read 0x2c4844f thru 0x2c4fdae (0x7960 bytes) 2024-09-13T09:26:05.382 INFO:teuthology.orchestra.run.smithi063.stdout:472 writesame 0x6980971 thru 0x6989e83 data_size 0x1(0x9513 bytes) 2024-09-13T09:26:05.406 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:05.413 INFO:teuthology.orchestra.run.smithi063.stdout:474 punch from 0x13f88b3 to 0x13fa46e, (0x1bbb bytes) 2024-09-13T09:26:05.419 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:05.413+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:05.427 INFO:teuthology.orchestra.run.smithi063.stdout:475 writesame 0x75dcb16 thru 0x75dffaf data_size 0x1(0x349a bytes) 2024-09-13T09:26:05.429 INFO:teuthology.orchestra.run.smithi063.stdout:476 read 0x57258c6 thru 0x572746f (0x1baa bytes) 2024-09-13T09:26:05.429 INFO:teuthology.orchestra.run.smithi063.stdout:477 write 0x5bea8f0 thru 0x5beaef8 (0x609 bytes) 2024-09-13T09:26:05.442 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:05.457 INFO:teuthology.orchestra.run.smithi063.stdout:478 writesame 0xdfcf6fc thru 0xdfd4d02 data_size 0x3(0x5607 bytes) 2024-09-13T09:26:05.457 INFO:teuthology.orchestra.run.smithi063.stdout:480 write 0xd12691 thru 0xd207f4 (0xe164 bytes) 2024-09-13T09:26:05.461 INFO:teuthology.orchestra.run.smithi063.stdout:482 punch from 0xdaa8e0f to 0xdab7fd1, (0xf1c2 bytes) 2024-09-13T09:26:05.461 INFO:teuthology.orchestra.run.smithi063.stdout:483 read 0xb6bdf9 thru 0xb7b8ec (0xfaf4 bytes) 2024-09-13T09:26:05.466 INFO:teuthology.orchestra.run.smithi063.stdout:484 trunc from 0xdfd4d03 to 0xb4311f9 2024-09-13T09:26:05.468 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:05.480 INFO:teuthology.orchestra.run.smithi063.stdout:486 writesame 0xab9b106 thru 0xaba09a8 data_size 0x1(0x58a3 bytes) 2024-09-13T09:26:05.493 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:05.494 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:05.500 INFO:teuthology.orchestra.run.smithi063.stdout:487 punch from 0x9e6146d to 0x9e6252b, (0x10be bytes) 2024-09-13T09:26:05.505 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:05.507 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:05.518 INFO:teuthology.orchestra.run.smithi063.stdout:488 trunc from 0xb4311f9 to 0x1721199 2024-09-13T09:26:05.520 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:05.514+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:05.533 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:05.541 INFO:teuthology.orchestra.run.smithi063.stdout:489 punch from 0x7c83d1 to 0x7d769a, (0xf2c9 bytes) 2024-09-13T09:26:05.553 INFO:teuthology.orchestra.run.smithi063.stdout:490 trunc from 0x1721199 to 0x83c6237 2024-09-13T09:26:05.561 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:05.561 INFO:teuthology.orchestra.run.smithi063.stdout:491 punch from 0x6df18fa to 0x6dfbff2, (0xa6f8 bytes) 2024-09-13T09:26:05.561 INFO:teuthology.orchestra.run.smithi063.stdout:492 write 0xdc574c thru 0xdd4da5 (0xf65a bytes) 2024-09-13T09:26:05.565 INFO:teuthology.orchestra.run.smithi063.stdout:494 write 0x6eb5dc6 thru 0x6ec48dc (0xeb17 bytes) 2024-09-13T09:26:05.571 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.571 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.571 INFO:teuthology.orchestra.run.smithi063.stdout:497 write 0x6805537 thru 0x680d040 (0x7b0a bytes) 2024-09-13T09:26:05.574 INFO:teuthology.orchestra.run.smithi063.stdout:498 read 0x5b8ddfd thru 0x5b9b892 (0xda96 bytes) 2024-09-13T09:26:05.575 INFO:teuthology.orchestra.run.smithi063.stdout:499 read 0x1da8cf9 thru 0x1daec96 (0x5f9e bytes) 2024-09-13T09:26:05.576 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.578 INFO:teuthology.orchestra.run.smithi063.stdout:501 trunc from 0x83c6237 to 0x9c7bb87 2024-09-13T09:26:05.588 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:05.589 INFO:teuthology.orchestra.run.smithi063.stdout:503 trunc from 0x9c7bb87 to 0xb42bd53 2024-09-13T09:26:05.591 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:05.591 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:05.591 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:05.591 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:05.592 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:05.592 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:05.592 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:05.592 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:05.601 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:05.601 INFO:teuthology.orchestra.run.smithi063.stdout:506 write 0x9905449 thru 0x9907b7a (0x2732 bytes) 2024-09-13T09:26:05.606 INFO:teuthology.orchestra.run.smithi063.stdout:507 clone 5 order 22 su 1048576 sc 3 2024-09-13T09:26:05.621 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:05.613+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:05.633 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:05.634 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:05.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:05.721 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:05.714+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:05.795 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:05.795 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:05.805 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:05.822 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:05.821+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:05.916 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:05.917 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:05.923 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:05.917+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:05.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:06.023 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:06.017+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:06.036 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.036 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:06.124 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:06.121+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:06.155 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.155 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:06.225 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:06.221+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:06.280 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.281 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:26:06.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:06.334 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:06.330+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:06.393 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone4 from 0xb42bd53 (overlap 0x1721199) to 0x14d0fd6 2024-09-13T09:26:06.427 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:06.421+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:06.472 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.472 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:06.499 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T09:26:06.528 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:06.522+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:06.601 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.601 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:06.628 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:06.621+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:06.721 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.721 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.729 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:06.722+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:06.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:06.831 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:06.825+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:06.833 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:06.833 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:06.843 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:06.880 INFO:teuthology.orchestra.run.smithi063.stdout:508 punch from 0x72f1778 to 0x72f6c83, (0x550b bytes) 2024-09-13T09:26:06.889 INFO:teuthology.orchestra.run.smithi063.stdout:509 write 0x6b406ab thru 0x6b4b10a (0xaa60 bytes) 2024-09-13T09:26:06.897 INFO:teuthology.orchestra.run.smithi063.stdout:510 writesame 0x3086083 thru 0x3095882 data_size 0x80(0xf800 bytes) 2024-09-13T09:26:06.904 INFO:teuthology.orchestra.run.smithi063.stdout:512 write 0x1e41f81 thru 0x1e47b15 (0x5b95 bytes) 2024-09-13T09:26:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:513 read 0x306fc64 thru 0x30739d8 (0x3d75 bytes) 2024-09-13T09:26:06.912 INFO:teuthology.orchestra.run.smithi063.stdout:514 writesame 0x5018422 thru 0x5019e12 data_size 0x1(0x19f1 bytes) 2024-09-13T09:26:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:515 punch from 0x51d197b to 0x51d4f68, (0x35ed bytes) 2024-09-13T09:26:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:516 punch from 0x3842172 to 0x384539e, (0x322c bytes) 2024-09-13T09:26:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:517 writesame 0x78a04c3 thru 0x78afedd data_size 0x1(0xfa1b bytes) 2024-09-13T09:26:06.932 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:06.925+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:06.957 INFO:teuthology.orchestra.run.smithi063.stdout:518 write 0x6a313e9 thru 0x6a3f97e (0xe596 bytes) 2024-09-13T09:26:06.965 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:06.965 INFO:teuthology.orchestra.run.smithi063.stdout:520 trunc from 0xb42bd53 to 0xb0e2416 2024-09-13T09:26:06.985 INFO:teuthology.orchestra.run.smithi063.stdout:522 read 0x5a767b thru 0x5b12ac (0x9c32 bytes) 2024-09-13T09:26:06.986 INFO:teuthology.orchestra.run.smithi063.stdout:524 read 0x91f5958 thru 0x920340e (0xdab7 bytes) 2024-09-13T09:26:06.987 INFO:teuthology.orchestra.run.smithi063.stdout:527 read 0x72e2c05 thru 0x72e9816 (0x6c12 bytes) 2024-09-13T09:26:06.989 INFO:teuthology.orchestra.run.smithi063.stdout:528 trunc from 0xb0e2416 to 0xc0014d0 2024-09-13T09:26:07.009 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.009 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:07.021 INFO:teuthology.orchestra.run.smithi063.stdout:529 writesame 0xd2452f2 thru 0xd254955 data_size 0x4(0xf664 bytes) 2024-09-13T09:26:07.025 INFO:teuthology.orchestra.run.smithi063.stdout:530 trunc from 0xd254956 to 0xec025d7 2024-09-13T09:26:07.033 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:07.026+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:07.056 INFO:teuthology.orchestra.run.smithi063.stdout:531 trunc from 0xec025d7 to 0x6252c4c 2024-09-13T09:26:07.094 INFO:teuthology.orchestra.run.smithi063.stdout:533 trunc from 0x6252c4c to 0x24cb6d2 2024-09-13T09:26:07.123 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.123 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.130 INFO:teuthology.orchestra.run.smithi063.stdout:534 trunc from 0x24cb6d2 to 0x693af36 2024-09-13T09:26:07.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:07.134 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:07.129+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:07.153 INFO:teuthology.orchestra.run.smithi063.stdout:535 trunc from 0x693af36 to 0xad2fe8 2024-09-13T09:26:07.212 INFO:teuthology.orchestra.run.smithi063.stdout:536 write 0xa3beb65 thru 0xa3bf7ca (0xc66 bytes) 2024-09-13T09:26:07.218 INFO:teuthology.orchestra.run.smithi063.stdout:537 writesame 0x684bad9 thru 0x6857dc2 data_size 0x3d(0xc2ea bytes) 2024-09-13T09:26:07.225 INFO:teuthology.orchestra.run.smithi063.stdout:539 read 0x21bfd13 thru 0x21cc0c8 (0xc3b6 bytes) 2024-09-13T09:26:07.225 INFO:teuthology.orchestra.run.smithi063.stdout:540 write 0x55e07d5 thru 0x55eed38 (0xe564 bytes) 2024-09-13T09:26:07.231 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:07.231 INFO:teuthology.orchestra.run.smithi063.stdout:542 punch from 0x67f0a61 to 0x67fdbde, (0xd17d bytes) 2024-09-13T09:26:07.231 INFO:teuthology.orchestra.run.smithi063.stdout:543 punch from 0x2d5f6f9 to 0x2d6d963, (0xe26a bytes) 2024-09-13T09:26:07.235 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:07.229+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:07.236 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.237 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:07.252 INFO:teuthology.orchestra.run.smithi063.stdout:544 trunc from 0xa3bf7cb to 0x22964e 2024-09-13T09:26:07.280 INFO:teuthology.orchestra.run.smithi063.stdout:545 read 0x2960d thru 0x2e221 (0x4c15 bytes) 2024-09-13T09:26:07.280 INFO:teuthology.orchestra.run.smithi063.stdout:546 read 0x1ab88c thru 0x1b21cc (0x6941 bytes) 2024-09-13T09:26:07.289 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:26:07.303 INFO:teuthology.orchestra.run.smithi063.stdout:547 write 0x539df0a thru 0x539ed73 (0xe6a bytes) 2024-09-13T09:26:07.333 INFO:teuthology.orchestra.run.smithi063.stdout:548 write 0xa282e74 thru 0xa287ab4 (0x4c41 bytes) 2024-09-13T09:26:07.336 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:07.330+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:07.339 INFO:teuthology.orchestra.run.smithi063.stdout:549 writesame 0x7ae7546 thru 0x7af0447 data_size 0x23(0x8f02 bytes) 2024-09-13T09:26:07.345 INFO:teuthology.orchestra.run.smithi063.stdout:551 punch from 0x5d32b7f to 0x5d3608b, (0x350c bytes) 2024-09-13T09:26:07.345 INFO:teuthology.orchestra.run.smithi063.stdout:552 writesame 0x7799551 thru 0x779d4a8 data_size 0x4(0x3f58 bytes) 2024-09-13T09:26:07.348 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:26:07.348 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:07.348 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-13T09:26:07.348 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:26:07.348 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:26:07.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-13T09:26:07.352 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.352 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.355 INFO:teuthology.orchestra.run.smithi063.stdout:553 writesame 0x27ce41b thru 0x27d117a data_size 0x16(0x2d60 bytes) 2024-09-13T09:26:07.361 INFO:teuthology.orchestra.run.smithi063.stdout:554 read 0x745104 thru 0x74acfe (0x5bfb bytes) 2024-09-13T09:26:07.362 INFO:teuthology.orchestra.run.smithi063.stdout:555 write 0x5b5a158 thru 0x5b5bcfd (0x1ba6 bytes) 2024-09-13T09:26:07.363 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:07.366 INFO:teuthology.orchestra.run.smithi063.stdout:559 clone 6 order 25 su 262144 sc 11 2024-09-13T09:26:07.437 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:07.430+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:07.448 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:07.449 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:26:07.449 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:07.465 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.465 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.474 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:07.538 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:07.530+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:07.577 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.577 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:07.605 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:07.639 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:07.633+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:07.695 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.695 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.704 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:07.749 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:07.745+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:07.758 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:07.779 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:07.805 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:07.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:07.841 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:07.834+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:07.849 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:07.858 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:07.858 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:07.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:07.871 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:07.900 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:07.922 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:07.924 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:07.925 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:07.941 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:07.934+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:07.977 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 25.012265372000002, 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.015875916999999, 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.607773397999999, 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0121125, 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 108 2024-09-13T09:26:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365507+0000", 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365505+0000", 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365518+0000", 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2456000000000001e-05 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365553+0000", 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5086000000000003e-05 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365567+0000", 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4187e-05 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371405+0000", 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058378960000000004 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371551+0000", 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014597300000000001 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.371737+0000", 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018655899999999999 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377392+0000", 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056547380000000003 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377484+0000", 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2261000000000003e-05 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377565+0000", 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0464999999999996e-05 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377619+0000", 2024-09-13T09:26:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4221999999999997e-05 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.607302284999999, 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014920113, 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365977+0000", 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365987+0000", 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366004+0000", 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.662e-05 2024-09-13T09:26:07.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366022+0000", 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7883000000000001e-05 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376091+0000", 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010069171999999999 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376246+0000", 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000155377 2024-09-13T09:26:07.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376548+0000", 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030137300000000002 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380798+0000", 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004250406 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380813+0000", 2024-09-13T09:26:07.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4357e-05 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380862+0000", 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9137000000000001e-05 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380898+0000", 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6158999999999997e-05 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.607243706999999, 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016811104, 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366035+0000", 2024-09-13T09:26:08.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366045+0000", 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0327e-05 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366062+0000", 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6483000000000001e-05 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366074+0000", 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377874+0000", 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011799983999999999 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378006+0000", 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013130100000000001 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378143+0000", 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137157 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380914+0000", 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002771281 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382760+0000", 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001846253 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382810+0000", 2024-09-13T09:26:08.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9483e-05 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382847+0000", 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7749000000000001e-05 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:08.008 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.607164005000001, 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018719919000000002, 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366115+0000", 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366313+0000", 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000198156 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366331+0000", 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7739000000000001e-05 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366345+0000", 2024-09-13T09:26:08.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4226000000000001e-05 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378684+0000", 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012339325 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378827+0000", 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014284700000000001 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.379116+0000", 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028930399999999999 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382675+0000", 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035590050000000001 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384568+0000", 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001892707 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384706+0000", 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013804099999999999 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384836+0000", 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012977900000000001 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.014 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.606917846999998, 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01866932, 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:08.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366361+0000", 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366372+0000", 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0857e-05 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366388+0000", 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366401+0000", 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.326e-05 2024-09-13T09:26:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380005+0000", 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013604036 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380100+0000", 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4468000000000004e-05 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380211+0000", 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011084199999999999 2024-09-13T09:26:08.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383662+0000", 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034510460000000001 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384905+0000", 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001243009 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384996+0000", 2024-09-13T09:26:08.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385032+0000", 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5528000000000003e-05 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.606864997999999, 2024-09-13T09:26:08.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020964449999999999, 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366414+0000", 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366423+0000", 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366443+0000", 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366455+0000", 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380966+0000", 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014510662000000001 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381062+0000", 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6526000000000004e-05 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381259+0000", 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019658499999999999 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383675+0000", 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024160480000000001 2024-09-13T09:26:08.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387295+0000", 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036198020000000001 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387342+0000", 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7299000000000002e-05 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387380+0000", 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7379000000000002e-05 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.606440971000001, 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020678247, 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366838+0000", 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367071+0000", 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023322800000000001 2024-09-13T09:26:08.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367094+0000", 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3079999999999999e-05 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367111+0000", 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7008999999999999e-05 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381788+0000", 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014676850999999999 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381894+0000", 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010610799999999999 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382045+0000", 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015073699999999999 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383689+0000", 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016436630000000001 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387445+0000", 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037564 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387488+0000", 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3402000000000002e-05 2024-09-13T09:26:08.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387517+0000", 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8975999999999998e-05 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.606151283999999, 2024-09-13T09:26:08.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020557408999999999, 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367128+0000", 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367138+0000", 2024-09-13T09:26:08.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0672999999999999e-05 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367157+0000", 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8779000000000001e-05 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367174+0000", 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6464e-05 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382926+0000", 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015752759000000002 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383077+0000", 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150965 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383228+0000", 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015101399999999999 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384458+0000", 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012294370000000001 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387599+0000", 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031411960000000002 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387647+0000", 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7648999999999997e-05 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387686+0000", 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9653000000000001e-05 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.606092179000001, 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022933500999999998, 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:08.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367187+0000", 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367208+0000", 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1793e-05 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367225+0000", 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668999999999999e-05 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367239+0000", 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4005999999999999e-05 2024-09-13T09:26:08.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383748+0000", 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016508719000000002 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383933+0000", 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018481500000000001 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384122+0000", 2024-09-13T09:26:08.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189626 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385049+0000", 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092678599999999997 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390003+0000", 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049541819999999997 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390075+0000", 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2094999999999993e-05 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390121+0000", 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6082999999999999e-05 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:134 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:26:08.044 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.016203171000001, 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012255703999999999, 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 134 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957077+0000", 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957076+0000", 2024-09-13T09:26:08.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957179+0000", 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103136 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957213+0000", 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4422999999999999e-05 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957225+0000", 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1178e-05 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963332+0000", 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061077019999999996 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963543+0000", 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021082699999999999 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.963933+0000", 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038984300000000001 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967237+0000", 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033042449999999999 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969266+0000", 2024-09-13T09:26:08.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020291200000000001 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969303+0000", 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6183000000000002e-05 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969333+0000", 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0063e-05 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.016036798999998, 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015204896000000001, 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 135 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957242+0000", 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957250+0000", 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957268+0000", 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7459999999999999e-05 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957277+0000", 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3559999999999999e-06 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965456+0000", 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792359999999995 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965567+0000", 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110877 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965741+0000", 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017349600000000001 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967304+0000", 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001563444 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972380+0000", 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050761019999999999 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972425+0000", 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4366000000000002e-05 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972448+0000", 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3455000000000002e-05 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.015992729000001, 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015208101, 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 136 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957297+0000", 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9909999999999997e-06 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957339+0000", 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2267e-05 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957371+0000", 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2518000000000003e-05 2024-09-13T09:26:08.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966255+0000", 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088832679999999997 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966353+0000", 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8658000000000003e-05 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966486+0000", 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013309500000000001 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969257+0000", 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027707209999999999 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972453+0000", 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031962789999999998 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972478+0000", 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4655e-05 2024-09-13T09:26:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972495+0000", 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7407000000000001e-05 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:08.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.015877868, 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015156871000000001, 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957401+0000", 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957509+0000", 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010808 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957536+0000", 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6237e-05 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957542+0000", 2024-09-13T09:26:08.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2020000000000001e-06 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967352+0000", 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098099580000000006 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967447+0000", 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5375000000000001e-05 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967584+0000", 2024-09-13T09:26:08.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136463 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971541+0000", 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039574930000000003 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972499+0000", 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095777799999999995 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972527+0000", 2024-09-13T09:26:08.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7900999999999999e-05 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972559+0000", 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2191999999999999e-05 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.015728030999998, 2024-09-13T09:26:08.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015104328, 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957551+0000", 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957556+0000", 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.318e-06 2024-09-13T09:26:08.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957565+0000", 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.115e-06 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957572+0000", 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968300+0000", 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010727617 2024-09-13T09:26:08.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968387+0000", 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7021000000000006e-05 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968482+0000", 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5081999999999995e-05 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971595+0000", 2024-09-13T09:26:08.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031126639999999998 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972588+0000", 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099296600000000007 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972635+0000", 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7136999999999999e-05 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972656+0000", 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1480000000000001e-05 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.015699229999999, 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015429492, 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.074 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957580+0000", 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957586+0000", 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957620+0000", 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3215e-05 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957651+0000", 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1027e-05 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969570+0000", 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011919511000000001 2024-09-13T09:26:08.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969674+0000", 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104248 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969784+0000", 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010937700000000001 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972662+0000", 2024-09-13T09:26:08.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028781589999999999 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972925+0000", 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026289899999999998 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972976+0000", 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0702999999999999e-05 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973010+0000", 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4743000000000003e-05 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.015594625999999, 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020098000000000001, 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957793+0000", 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108837 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957808+0000", 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957814+0000", 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970251+0000", 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012436523999999999 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970350+0000", 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9600999999999997e-05 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970506+0000", 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015539500000000001 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972667+0000", 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002161619 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977681+0000", 2024-09-13T09:26:08.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050135930000000002 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977722+0000", 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1634999999999997e-05 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977783+0000", 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0968000000000003e-05 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.015458499000001, 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022700502000000001, 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-13T09:26:08.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957821+0000", 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957825+0000", 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957843+0000", 2024-09-13T09:26:08.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7492000000000001e-05 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957851+0000", 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971666+0000", 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013814707000000001 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971841+0000", 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174659 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971976+0000", 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013485600000000001 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973161+0000", 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011852130000000001 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980427+0000", 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072666440000000001 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980487+0000", 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9373000000000002e-05 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980522+0000", 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5244999999999999e-05 2024-09-13T09:26:08.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.197590570999999, 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013540165999999999, 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:08.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775689+0000", 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775694+0000", 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8520000000000003e-06 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:08.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775699+0000", 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.682e-06 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.775709+0000", 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3980000000000005e-06 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783188+0000", 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074792560000000001 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783392+0000", 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020345199999999999 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.783653+0000", 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026178699999999998 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.785173+0000", 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001519739 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789105+0000", 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039318900000000004 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789174+0000", 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9568999999999997e-05 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:45.789230+0000", 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5169e-05 2024-09-13T09:26:08.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:08.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:08.099 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:08.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:08.101 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:08.094+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:08.201 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:08.194+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:08.219 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:08.219 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:08.219 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:08.219 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:08.229 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:08.302 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:08.297+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:08.342 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:08.342 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:08.342 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:08.342 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:08.342 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:08.353 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:08.403 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:08.398+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:08.433 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone5 from 0xa287ab5 (overlap 0x22964e) to 0x114b27 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:08.459 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.505142512999999, 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:08.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:08.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.488584299999999, 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:08.467 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.253837042000001, 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.473 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.251627904999999, 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:08.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.247283380000001, 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:08.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.230732400999999, 2024-09-13T09:26:08.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:08.486 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.230144393, 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.222438237, 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.491 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:08.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.495 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:08.496 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:08.538 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:08.534+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:08.578 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T09:26:08.604 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:08.598+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:08.643 INFO:teuthology.orchestra.run.smithi063.stdout:561 write 0x5312a69 thru 0x531ef0a (0xc4a2 bytes) 2024-09-13T09:26:08.653 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:08.654 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:08.654 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:08.654 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:08.661 INFO:teuthology.orchestra.run.smithi063.stdout:565 clone 7 order 25 su 2097152 sc 9 2024-09-13T09:26:08.664 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:08.706 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:08.701+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:08.764 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:08.764 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:08.764 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:08.764 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:08.764 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:08.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:08.806 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:08.801+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4211.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:08.886 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 25.931222218999999, 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051933830000000002, 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4211", 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 1 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948437+0000", 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948452+0000", 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4372999999999999e-05 2024-09-13T09:26:08.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948471+0000", 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8932000000000001e-05 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948488+0000", 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7002000000000001e-05 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948542+0000", 2024-09-13T09:26:08.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4128000000000001e-05 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948551+0000", 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1239999999999994e-06 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.951931+0000", 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003380245 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.952258+0000", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032729100000000001 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.952439+0000", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018074399999999999 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953396+0000", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00095699900000000004 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953544+0000", 2024-09-13T09:26:08.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014766599999999999 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953591+0000", 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953634+0000", 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:55 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'33, mlcod=18'29)", 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.103977+0000", 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.775685525, 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048140429999999996, 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.103977+0000", 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.103977+0000", 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.103975+0000", 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104032+0000", 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6483000000000001e-05 2024-09-13T09:26:08.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104063+0000", 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0839000000000001e-05 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104075+0000", 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2303000000000001e-05 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104157+0000", 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1951000000000002e-05 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.104199+0000", 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1766e-05 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.108684+0000", 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044854600000000001 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.108791+0000", 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010694800000000001 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:56 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'34, mlcod=18'29)", 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.774116907, 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005743209, 2024-09-13T09:26:08.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105544+0000", 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105557+0000", 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3464e-05 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105607+0000", 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9941999999999997e-05 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105620+0000", 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3049e-05 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105726+0000", 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105185 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105793+0000", 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7164999999999998e-05 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.111223+0000", 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054298690000000004 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.111289+0000", 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6074999999999993e-05 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:112 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'74, mlcod=18'70)", 2024-09-13T09:26:08.901 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.376909+0000", 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.502753109, 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049794069999999999, 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376909+0000", 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376909+0000", 2024-09-13T09:26:08.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376908+0000", 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376968+0000", 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0851999999999999e-05 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.376998+0000", 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9346e-05 2024-09-13T09:26:08.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377010+0000", 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2119999999999999e-05 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377111+0000", 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101004 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.377150+0000", 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9449999999999997e-05 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381831+0000", 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046802249999999997 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381889+0000", 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8122999999999999e-05 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:114 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'76, mlcod=18'71)", 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.500697083999999, 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005632972, 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378964+0000", 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378976+0000", 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1652e-05 2024-09-13T09:26:08.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378990+0000", 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.399e-05 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378999+0000", 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1349999999999998e-06 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379073+0000", 2024-09-13T09:26:08.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4256000000000002e-05 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379142+0000", 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9139000000000003e-05 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384494+0000", 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053517829999999997 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384598+0000", 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010432 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:115 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'77, mlcod=18'71)", 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.379781+0000", 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.499881109, 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048443180000000002, 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379781+0000", 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379781+0000", 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379780+0000", 2024-09-13T09:26:08.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379858+0000", 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8008000000000003e-05 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379876+0000", 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7649999999999999e-05 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379884+0000", 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6199999999999999e-06 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379954+0000", 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0223999999999997e-05 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379985+0000", 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1251000000000003e-05 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384602+0000", 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046171019999999997 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384626+0000", 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3428999999999999e-05 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:117 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'79, mlcod=18'72)", 2024-09-13T09:26:08.914 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.497810005000002, 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054346220000000001, 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381851+0000", 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381865+0000", 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4103e-05 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381883+0000", 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8522999999999999e-05 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381909+0000", 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5712999999999999e-05 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381989+0000", 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0086000000000005e-05 2024-09-13T09:26:08.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.382020+0000", 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0712e-05 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.387223+0000", 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052033599999999998 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.387287+0000", 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3721999999999997e-05 2024-09-13T09:26:08.918 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:140 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'98, mlcod=18'91)", 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.911491171000002, 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0068715920000000002, 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:26:08.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968170+0000", 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968240+0000", 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9692999999999998e-05 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968254+0000", 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4268e-05 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968260+0000", 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968307+0000", 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6836999999999999e-05 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968323+0000", 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6494000000000002e-05 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.974993+0000", 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066696510000000004 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.975043+0000", 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0002e-05 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:141 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'99, mlcod=18'91)", 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.909943707, 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080457759999999993, 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.923 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969718+0000", 2024-09-13T09:26:08.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969726+0000", 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969754+0000", 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8481000000000001e-05 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969764+0000", 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6509999999999996e-06 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969818+0000", 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4425000000000002e-05 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969848+0000", 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0122000000000001e-05 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.977718+0000", 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078694349999999993 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.977765+0000", 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6739000000000001e-05 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:144 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'102, mlcod=19'98)", 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.899130666000001, 2024-09-13T09:26:08.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058040629999999999, 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980531+0000", 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980635+0000", 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010367800000000001 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980690+0000", 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5795999999999999e-05 2024-09-13T09:26:08.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980702+0000", 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1788e-05 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980759+0000", 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7141000000000001e-05 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980834+0000", 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4836e-05 2024-09-13T09:26:08.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986281+0000", 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054468370000000004 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986336+0000", 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.931 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:145 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'103, mlcod=19'98)", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.981315+0000", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.898347509000001, 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050415989999999999, 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981315+0000", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981315+0000", 2024-09-13T09:26:08.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981314+0000", 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981396+0000", 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2196999999999999e-05 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981427+0000", 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1165999999999999e-05 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981445+0000", 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8236999999999998e-05 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981494+0000", 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8520000000000003e-05 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.981526+0000", 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1588999999999999e-05 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986340+0000", 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048145640000000003 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986357+0000", 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6359999999999999e-05 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.935 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:159 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'113, mlcod=19'107)", 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.098342973000001, 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051229429999999996, 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781319+0000", 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781327+0000", 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4819999999999994e-06 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781347+0000", 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0279e-05 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781361+0000", 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3587999999999999e-05 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781410+0000", 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9589999999999998e-05 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.781422+0000", 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1718e-05 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.786363+0000", 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049406010000000002 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:45.786442+0000", 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9585000000000005e-05 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:276 3.4 3:27927849:::rbd_data.111867511a81.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.503158+0000", 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3765043640000005, 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044520039999999999, 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 276 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503158+0000", 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503158+0000", 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503156+0000", 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503173+0000", 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7280000000000001e-05 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503202+0000", 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9036000000000001e-05 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503224+0000", 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2269999999999999e-05 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503385+0000", 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161006 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503717+0000", 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033205300000000001 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504077+0000", 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000359995 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.506899+0000", 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028216690000000002 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507466+0000", 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00056658700000000002 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507567+0000", 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010133599999999999 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507610+0000", 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3195000000000002e-05 2024-09-13T09:26:08.945 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:340 3.6 3:66b996d9:::rbd_data.111867511a81.0000000000000023:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.709723+0000", 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.1699395629999998, 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047902409999999998, 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 340 2024-09-13T09:26:08.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709723+0000", 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709723+0000", 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709720+0000", 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:08.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709739+0000", 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9803e-05 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709785+0000", 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5188999999999997e-05 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709813+0000", 2024-09-13T09:26:08.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8379000000000001e-05 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.709955+0000", 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014190000000000001 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.710253+0000", 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00029786500000000002 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:08.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.710932+0000", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00067924700000000001 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.712169+0000", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001236766 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.714412+0000", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002242979 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.714469+0000", 2024-09-13T09:26:08.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7003999999999998e-05 2024-09-13T09:26:08.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.714513+0000", 2024-09-13T09:26:08.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4362e-05 2024-09-13T09:26:08.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:385 3.0 3:0220607e:::rbd_header.111867511a81:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802096+0000", 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0775663140000002, 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004604225, 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.952 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 385 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802096+0000", 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802096+0000", 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802093+0000", 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802111+0000", 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8379000000000001e-05 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802150+0000", 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8971999999999999e-05 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802166+0000", 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5434000000000002e-05 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802241+0000", 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5203999999999994e-05 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802401+0000", 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016030600000000001 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:08.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802696+0000", 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00029506399999999999 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803704+0000", 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010082769999999999 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806638+0000", 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029331499999999998 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806673+0000", 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5621000000000003e-05 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806700+0000", 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7140000000000001e-05 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.957 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:1872 3.6 3:679e641d:::rbd_data.1118e3e5e25d.00000000000000d1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.089768+0000", 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7898939719999998, 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043255760000000002, 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 1872 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089768+0000", 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089768+0000", 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089768+0000", 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089774+0000", 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8129999999999998e-06 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089791+0000", 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6671999999999999e-05 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089798+0000", 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089864+0000", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6023000000000001e-05 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.089960+0000", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6082000000000006e-05 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.090094+0000", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013410000000000001 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.090720+0000", 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00062536700000000005 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.094036+0000", 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003316538 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.094070+0000", 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3742000000000003e-05 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.094094+0000", 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3937000000000001e-05 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2048 3.8 e30/21 3:119bd4bc:::rbd_data.1118e3e5e25d.000000000000011f:head v 30'13, mlcod=30'11)", 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.202572+0000", 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.6770902809999999, 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049782469999999999, 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202572+0000", 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202572+0000", 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202571+0000", 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202582+0000", 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1433999999999999e-05 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202604+0000", 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2031999999999998e-05 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202614+0000", 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7089999999999997e-06 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202668+0000", 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4308999999999998e-05 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202696+0000", 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7234e-05 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207491+0000", 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047957499999999997 2024-09-13T09:26:08.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207550+0000", 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9067999999999998e-05 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2061 3.4 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head [stat,truncate 98304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.210496+0000", 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.6691662279999999, 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004367442, 2024-09-13T09:26:08.967 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2061 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210496+0000", 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210496+0000", 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210495+0000", 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210506+0000", 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.146e-05 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210523+0000", 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7102e-05 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210532+0000", 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2430000000000005e-06 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210604+0000", 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2733999999999999e-05 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210748+0000", 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014339499999999999 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.210956+0000", 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020844099999999999 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214409+0000", 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003453242 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214744+0000", 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033452900000000002 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214804+0000", 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0170999999999997e-05 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214864+0000", 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9568999999999998e-05 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2414 3.4 3:232a72f0:::rbd_header.1118e3e5e25d:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:04.262348+0000", 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6173140149999998, 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042299520000000004, 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2414 2024-09-13T09:26:08.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262348+0000", 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262348+0000", 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262345+0000", 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:08.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262365+0000", 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9408000000000001e-05 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262402+0000", 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7936999999999999e-05 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262419+0000", 2024-09-13T09:26:08.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6762e-05 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262588+0000", 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016902 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.262883+0000", 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00029494999999999999 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:08.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264136+0000", 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001253225 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.265358+0000", 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001221537 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.266373+0000", 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00101458 2024-09-13T09:26:08.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.266458+0000", 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5749999999999997e-05 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.266578+0000", 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012006300000000001 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:3639 3.6 3:606ff0dc:::rbd_header.11189888a71f:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:08.345983+0000", 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.53367971400000003, 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043005370000000001, 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 3639 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:08.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.345983+0000", 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.345983+0000", 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.345980+0000", 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.345997+0000", 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7028e-05 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.346024+0000", 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6801999999999999e-05 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.346038+0000", 2024-09-13T09:26:08.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3665999999999999e-05 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.346194+0000", 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015625599999999999 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.346210+0000", 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5733e-05 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.349647+0000", 2024-09-13T09:26:08.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034373530000000002 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.349874+0000", 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022666900000000001 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.350283+0000", 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00040956400000000001 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.983 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.984 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:08.984 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:08.984 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:08.985 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:08.986 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:08.977+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:09.086 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:09.078+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:09.098 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:09.098 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:09.098 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:09.099 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:09.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:09.187 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:09.181+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:09.261 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:09.262 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:09.262 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:09.262 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:09.262 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:09.271 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:09.287 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:09.281+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:09.303 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:26:09.388 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:09.382+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:276 3.4 e27/21 3:27927849:::rbd_data.111867511a81.0000000000000026:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.8821824100000004, 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002929961, 2024-09-13T09:26:09.392 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504324+0000", 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504366+0000", 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2412e-05 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504407+0000", 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0329e-05 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504427+0000", 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0089999999999999e-05 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504518+0000", 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0801e-05 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504566+0000", 2024-09-13T09:26:09.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8745999999999997e-05 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507192+0000", 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00262546 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507256+0000", 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.396 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:307 3.9 e27/21 3:92c4c1be:::rbd_data.111867511a81.0000000000000087:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.7569359379999998, 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002303486, 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:09.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629570+0000", 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629585+0000", 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4826e-05 2024-09-13T09:26:09.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629615+0000", 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9879e-05 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629628+0000", 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2673e-05 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629749+0000", 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121304 2024-09-13T09:26:09.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629816+0000", 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7285000000000004e-05 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.631763+0000", 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019471040000000001 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.631876+0000", 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112151 2024-09-13T09:26:09.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.5835694149999995, 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:26:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:1784 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'50, mlcod=30'48)", 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:02.977252+0000", 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.4092563179999997, 2024-09-13T09:26:09.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001615815, 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977252+0000", 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977252+0000", 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977250+0000", 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977289+0000", 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8560999999999999e-05 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977310+0000", 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0998000000000001e-05 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977320+0000", 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977425+0000", 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105855 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977478+0000", 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2966000000000001e-05 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.978823+0000", 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013447820000000001 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.978867+0000", 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4326000000000002e-05 2024-09-13T09:26:09.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:1785 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'7, mlcod=30'5)", 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:02.977465+0000", 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.4090427959999996, 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001946897, 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977465+0000", 2024-09-13T09:26:09.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977465+0000", 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977464+0000", 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977597+0000", 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013270900000000001 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977608+0000", 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0831e-05 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977616+0000", 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977687+0000", 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0319000000000002e-05 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.977731+0000", 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4743000000000002e-05 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.979351+0000", 2024-09-13T09:26:09.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016195579999999999 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.979412+0000", 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1030999999999999e-05 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:1854 3.1f e30/25 3:fab6d9be:::rbd_data.1118e3e5e25d.0000000000000096:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.081823+0000", 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.304685192, 2024-09-13T09:26:09.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001928087, 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.081823+0000", 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.081823+0000", 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.081821+0000", 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.081868+0000", 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7667000000000003e-05 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.081906+0000", 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7370999999999998e-05 2024-09-13T09:26:09.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.081924+0000", 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8851e-05 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.082067+0000", 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000142965 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.082129+0000", 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1252000000000002e-05 2024-09-13T09:26:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.083679+0000", 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015502420000000001 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.083751+0000", 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1981000000000001e-05 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:1864 3.0 e30/21 3:01b4d536:::rbd_data.1118e3e5e25d.00000000000000c9:head v 30'41, mlcod=30'39)", 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.085544+0000", 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.300963694, 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019299499999999999, 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085544+0000", 2024-09-13T09:26:09.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085544+0000", 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085543+0000", 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085557+0000", 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4911e-05 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085586+0000", 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8302000000000001e-05 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085606+0000", 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0499000000000001e-05 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085674+0000", 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7764999999999999e-05 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.085694+0000", 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.087412+0000", 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017180260000000001 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.087474+0000", 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1767999999999995e-05 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2032 3.9 e30/21 3:92558f7b:::rbd_data.1118e3e5e25d.000000000000016f:head v 30'10, mlcod=30'8)", 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.176443+0000", 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2100647139999996, 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001766423, 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.423 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176443+0000", 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176443+0000", 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176442+0000", 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176482+0000", 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0092000000000002e-05 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176507+0000", 2024-09-13T09:26:09.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4465999999999999e-05 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176516+0000", 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8209999999999997e-06 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176636+0000", 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000119706 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.176678+0000", 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1798999999999997e-05 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.178135+0000", 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014567479999999999 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.178210+0000", 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5080999999999995e-05 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2044 3.4 e30/21 3:24072535:::rbd_data.1118e3e5e25d.0000000000000018:head v 30'59, mlcod=30'57)", 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.201188+0000", 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.1853202400000002, 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017393689999999999, 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201188+0000", 2024-09-13T09:26:09.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201188+0000", 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201186+0000", 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201211+0000", 2024-09-13T09:26:09.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4652999999999999e-05 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201244+0000", 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3685e-05 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201261+0000", 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6977e-05 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201430+0000", 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016834799999999999 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.201490+0000", 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0483999999999999e-05 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202840+0000", 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001349278 2024-09-13T09:26:09.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202927+0000", 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7581000000000001e-05 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2051 3.9 e30/21 3:92558f7b:::rbd_data.1118e3e5e25d.000000000000016f:head v 30'11, mlcod=30'9)", 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.203176+0000", 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.1833314660000003, 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001824757, 2024-09-13T09:26:09.432 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203176+0000", 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203176+0000", 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203175+0000", 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203182+0000", 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9110000000000004e-06 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203196+0000", 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3532e-05 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203208+0000", 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1919000000000001e-05 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203279+0000", 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0654000000000005e-05 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203314+0000", 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5620000000000001e-05 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.204939+0000", 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016252269999999999 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.205001+0000", 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1890999999999994e-05 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.436 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.17539353, 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:09.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:26:09.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.1752034069999997, 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:26:09.441 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:26:09.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.1743909979999998, 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:26:09.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:26:09.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2249 3.1f e30/25 3:fba30c3b:::rbd_data.1118e3e5e25d.0000000000000115:head v 30'16, mlcod=30'14)", 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.429214+0000", 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.9572942270000002, 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001666732, 2024-09-13T09:26:09.450 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429214+0000", 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429214+0000", 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429212+0000", 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429322+0000", 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109555 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429344+0000", 2024-09-13T09:26:09.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2532999999999999e-05 2024-09-13T09:26:09.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429355+0000", 2024-09-13T09:26:09.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1382999999999999e-05 2024-09-13T09:26:09.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429421+0000", 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5159000000000004e-05 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.429449+0000", 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8410000000000001e-05 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.430811+0000", 2024-09-13T09:26:09.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013619870000000001 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.430880+0000", 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9431000000000007e-05 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2370 3.9 e30/21 3:922e764a:::rbd_data.1118e3e5e25d.000000000000004f:head v 30'17, mlcod=30'15)", 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.695812+0000", 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.6906962720000003, 2024-09-13T09:26:09.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001618558, 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695812+0000", 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695812+0000", 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695810+0000", 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695830+0000", 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9335999999999999e-05 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695850+0000", 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0077000000000001e-05 2024-09-13T09:26:09.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695859+0000", 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8980000000000002e-06 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695956+0000", 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7423999999999994e-05 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.695994+0000", 2024-09-13T09:26:09.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8169e-05 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.697379+0000", 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013843029999999999 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.697430+0000", 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1625000000000001e-05 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2414 3.4 e31/21 3:232a72f0:::rbd_header.1118e3e5e25d:head v 31'76, mlcod=30'74)", 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:04.264473+0000", 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.1220348099999997, 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001628043, 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264473+0000", 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264473+0000", 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264473+0000", 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264483+0000", 2024-09-13T09:26:09.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0233e-05 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264512+0000", 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9655999999999999e-05 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264531+0000", 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8097e-05 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264661+0000", 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013041699999999999 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.264697+0000", 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5852000000000002e-05 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.266063+0000", 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001366017 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.266101+0000", 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8368000000000003e-05 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2766 3.1f e31/25 3:fd1e76d9:::rbd_data.11189888a71f.0000000000000002:head v 31'19, mlcod=30'17)", 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:04.834876+0000", 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.5516324240000001, 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015942549999999999, 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.834876+0000", 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.834876+0000", 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.834874+0000", 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.834887+0000", 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2568000000000001e-05 2024-09-13T09:26:09.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.834913+0000", 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6182000000000001e-05 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.834929+0000", 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5704000000000001e-05 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.835037+0000", 2024-09-13T09:26:09.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108491 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.835082+0000", 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4804e-05 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.836412+0000", 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001329623 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.836470+0000", 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8173e-05 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2983 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'24, mlcod=31'22)", 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.243457+0000", 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.1430514460000003, 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020652140000000001, 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243457+0000", 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243457+0000", 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243455+0000", 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243592+0000", 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013771700000000001 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243624+0000", 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1983999999999997e-05 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243639+0000", 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.518e-05 2024-09-13T09:26:09.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243738+0000", 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8302999999999997e-05 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243784+0000", 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6385999999999999e-05 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.245394+0000", 2024-09-13T09:26:09.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001609542 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.245522+0000", 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012801699999999999 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:26:09.473 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.985484349, 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:26:09.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3708 3.9 e34/21 3:92235de7:::rbd_data.111851704b2b.0000000000000027:head v 34'42, mlcod=32'39)", 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:08.438034+0000", 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.94847431000000004, 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021117089999999998, 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438034+0000", 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438034+0000", 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438033+0000", 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438043+0000", 2024-09-13T09:26:09.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3899999999999999e-06 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438069+0000", 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438087+0000", 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7286e-05 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438197+0000", 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109978 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438235+0000", 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8621999999999997e-05 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.440048+0000", 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018124269999999999 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.440145+0000", 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7733999999999996e-05 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.482 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:09.483 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:09.484 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone6 from 0xa287ab5 (overlap 0xa287ab5) to 0x71c55e5 2024-09-13T09:26:09.527 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:09.522+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:09.589 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:09.582+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:09.637 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:09.637 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:09.637 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:09.637 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:09.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:09.691 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:09.685+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:09.698 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T09:26:09.742 INFO:teuthology.orchestra.run.smithi063.stdout:566 write 0xbabf540 thru 0xbacca91 (0xd552 bytes) 2024-09-13T09:26:09.749 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:09.750 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:09.750 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:09.750 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:09.750 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:09.756 INFO:teuthology.orchestra.run.smithi063.stdout:567 read 0x2d9f6d1 thru 0x2da9f22 (0xa852 bytes) 2024-09-13T09:26:09.757 INFO:teuthology.orchestra.run.smithi063.stdout:568 read 0x3585c7c thru 0x358699d (0xd22 bytes) 2024-09-13T09:26:09.759 INFO:teuthology.orchestra.run.smithi063.stdout:569 write 0x8f3e2bc thru 0x8f4a5a1 (0xc2e6 bytes) 2024-09-13T09:26:09.761 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:09.782 INFO:teuthology.orchestra.run.smithi063.stdout:570 write 0xc449d9f thru 0xc44d3e0 (0x3642 bytes) 2024-09-13T09:26:09.785 INFO:teuthology.orchestra.run.smithi063.stdout:572 punch from 0x7388e31 to 0x738ad73, (0x1f42 bytes) 2024-09-13T09:26:09.785 INFO:teuthology.orchestra.run.smithi063.stdout:573 read 0xbbb6ffc thru 0xbbc3df7 (0xcdfc bytes) 2024-09-13T09:26:09.792 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:09.786+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:574 write 0xd2def3a thru 0xd2e005d (0x1124 bytes) 2024-09-13T09:26:09.811 INFO:teuthology.orchestra.run.smithi063.stdout:575 write 0xa8744f9 thru 0xa874527 (0x2f bytes) 2024-09-13T09:26:09.826 INFO:teuthology.orchestra.run.smithi063.stdout:577 writesame 0xd98e4aa thru 0xd993c9b data_size 0x1(0x57f2 bytes) 2024-09-13T09:26:09.872 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:12 3.18 3:1a426f9e:::rbd_header.11182e510213:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.661092754, 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0064571740000000004, 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 12 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204334+0000", 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204345+0000", 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0258000000000001e-05 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204358+0000", 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3867e-05 2024-09-13T09:26:09.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204372+0000", 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3754000000000001e-05 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204436+0000", 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204440+0000", 2024-09-13T09:26:09.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207020+0000", 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025802910000000002 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207135+0000", 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114299 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207374+0000", 2024-09-13T09:26:09.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023949400000000001 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208859+0000", 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014852209999999999 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210662+0000", 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001802894 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210738+0000", 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5448e-05 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210793+0000", 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5628999999999999e-05 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:272 3.7 3:e6b5b829:::rbd_data.111867511a81.0000000000000022:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:09.879 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.501686+0000", 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.363743283, 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055679329999999997, 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 272 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501686+0000", 2024-09-13T09:26:09.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501686+0000", 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501682+0000", 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501708+0000", 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5619e-05 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501759+0000", 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1048999999999999e-05 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501779+0000", 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0129999999999999e-05 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501923+0000", 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000144506 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.502250+0000", 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032710600000000002 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.502779+0000", 2024-09-13T09:26:09.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00052848499999999998 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.505619+0000", 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028406429999999999 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507069+0000", 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001449866 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507171+0000", 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010127 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507254+0000", 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3004999999999996e-05 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.885 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:274 3.7 3:e1d7b6d3:::rbd_data.111867511a81.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.501841+0000", 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.36358749, 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062127679999999996, 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 274 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501841+0000", 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501841+0000", 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501840+0000", 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501852+0000", 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2483000000000001e-05 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501872+0000", 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0095999999999999e-05 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501894+0000", 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1716e-05 2024-09-13T09:26:09.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503657+0000", 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001762668 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504024+0000", 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036737199999999998 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:09.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504720+0000", 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069563999999999997 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.506128+0000", 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014082789999999999 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507873+0000", 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017447719999999999 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507969+0000", 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6551000000000004e-05 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.508054+0000", 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4857000000000004e-05 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:421 3.8 3:11e50e73:::rbd_data.111867511a81.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:09.891 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.880682+0000", 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9847467610000002, 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056711770000000003, 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 421 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880682+0000", 2024-09-13T09:26:09.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880682+0000", 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880680+0000", 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880694+0000", 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3807e-05 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880726+0000", 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1628999999999998e-05 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880740+0000", 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880826+0000", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6417999999999998e-05 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.881013+0000", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000187269 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.881727+0000", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071376000000000002 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.885596+0000", 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038691670000000002 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.886198+0000", 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00060181399999999999 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.886286+0000", 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7887999999999996e-05 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.886353+0000", 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7402000000000003e-05 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:448 3.8 3:16185ba7:::rbd_data.111867511a81.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9126021069999997, 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022114892000000001, 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 448 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952824+0000", 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952840+0000", 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5420000000000001e-05 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952875+0000", 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5076000000000001e-05 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952890+0000", 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5596000000000001e-05 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.953033+0000", 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014238400000000001 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.953282+0000", 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024892700000000002 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.959632+0000", 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.006350372 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974849+0000", 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015217229000000001 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974877+0000", 2024-09-13T09:26:09.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7421999999999999e-05 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974914+0000", 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7320000000000002e-05 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974942+0000", 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7763000000000002e-05 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:524 3.1e 3:7c1f840a:::rbd_data.111867511a81.00000000000000d0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:00.164928+0000", 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.7005011200000002, 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063352649999999996, 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 524 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164928+0000", 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164928+0000", 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164925+0000", 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164943+0000", 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8182e-05 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164982+0000", 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8271e-05 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164998+0000", 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6639000000000001e-05 2024-09-13T09:26:09.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.165093+0000", 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5209000000000003e-05 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.165101+0000", 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0700000000000007e-06 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.169355+0000", 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042532380000000003 2024-09-13T09:26:09.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.169531+0000", 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017661399999999999 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.169748+0000", 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021657999999999999 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.170613+0000", 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00086520100000000001 2024-09-13T09:26:09.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.171135+0000", 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00052221899999999998 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.171183+0000", 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7420999999999999e-05 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.171263+0000", 2024-09-13T09:26:09.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0317999999999999e-05 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:1798 3.8 3:1362e7cf:::rbd_data.1118e3e5e25d.00000000000002ae:head [set-alloc-hint object_size 262144 write_size 262144,write 244159~17985] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:02.997512+0000", 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.8679173430000002, 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059031409999999998, 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.909 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 1798 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997512+0000", 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997512+0000", 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997511+0000", 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997523+0000", 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2473000000000001e-05 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997548+0000", 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5377000000000001e-05 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997554+0000", 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9340000000000003e-06 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997595+0000", 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0147e-05 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997672+0000", 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7785000000000001e-05 2024-09-13T09:26:09.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997757+0000", 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4976999999999996e-05 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.000666+0000", 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029086450000000001 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.003287+0000", 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026208550000000001 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.003341+0000", 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4293999999999999e-05 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.003415+0000", 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3578000000000006e-05 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2047 3.b 3:d68ef2c4:::rbd_data.1118e3e5e25d.000000000000001b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.202051+0000", 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.6633779720000001, 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056841720000000004, 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2047 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202051+0000", 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202051+0000", 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202049+0000", 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202063+0000", 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4114000000000001e-05 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202103+0000", 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0541999999999999e-05 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202122+0000", 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8499e-05 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202262+0000", 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014035100000000001 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202419+0000", 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015647400000000001 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:09.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202549+0000", 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013001399999999999 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203917+0000", 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013679639999999999 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207624+0000", 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037072659999999999 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207679+0000", 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4552000000000002e-05 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207735+0000", 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6541e-05 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2046 3.15 e30/21 3:afd6a079:::rbd_data.1118e3e5e25d.000000000000001a:head v 30'7, mlcod=30'5)", 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.202076+0000", 2024-09-13T09:26:09.920 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.6633533409999997, 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055546900000000001, 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202076+0000", 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202076+0000", 2024-09-13T09:26:09.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202073+0000", 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202091+0000", 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7893e-05 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202116+0000", 2024-09-13T09:26:09.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5106e-05 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202133+0000", 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7039999999999999e-05 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202354+0000", 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000221158 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202413+0000", 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8641999999999999e-05 2024-09-13T09:26:09.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207578+0000", 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051656250000000001 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207630+0000", 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2098999999999997e-05 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2048 3.8 3:119bd4bc:::rbd_data.1118e3e5e25d.000000000000011f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.202159+0000", 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.6632698939999999, 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005693048, 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2048 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.925 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202159+0000", 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202159+0000", 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202158+0000", 2024-09-13T09:26:09.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202167+0000", 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.363e-06 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202180+0000", 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.272e-05 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202188+0000", 2024-09-13T09:26:09.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4849999999999992e-06 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202257+0000", 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9160999999999997e-05 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202364+0000", 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010705300000000001 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:09.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202486+0000", 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012129999999999999 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.204934+0000", 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024484620000000002 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207718+0000", 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002783846 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207794+0000", 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6457000000000003e-05 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207852+0000", 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7572999999999999e-05 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2259 3.7 3:e2f2ee12:::rbd_data.1118e3e5e25d.00000000000001d1:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.464087+0000", 2024-09-13T09:26:09.930 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.4013423190000003, 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015117423, 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2259 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464087+0000", 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464087+0000", 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464084+0000", 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464097+0000", 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.324e-05 2024-09-13T09:26:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464127+0000", 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9742000000000001e-05 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464140+0000", 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3405e-05 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464255+0000", 2024-09-13T09:26:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115028 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464449+0000", 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019390600000000001 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.467089+0000", 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026396050000000002 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479136+0000", 2024-09-13T09:26:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012046708 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479148+0000", 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2415000000000001e-05 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479178+0000", 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0164000000000001e-05 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479204+0000", 2024-09-13T09:26:09.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5797999999999999e-05 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2258 3.17 e30/25 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head v 30'6, mlcod=29'4)", 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.470082+0000", 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.3953469939999996, 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00905918, 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.470082+0000", 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.470082+0000", 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.470080+0000", 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471660+0000", 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015793350000000001 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471693+0000", 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3676000000000001e-05 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471709+0000", 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5832999999999999e-05 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471783+0000", 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4203999999999996e-05 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471827+0000", 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3381999999999999e-05 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479105+0000", 2024-09-13T09:26:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0072785819999999996 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479141+0000", 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5921999999999999e-05 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2265 3.2 3:43c4f885:::rbd_data.1118e3e5e25d.000000000000019f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.3757775250000002, 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017493669999999999, 2024-09-13T09:26:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2265 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489650+0000", 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489658+0000", 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4939999999999997e-06 2024-09-13T09:26:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489675+0000", 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7384000000000002e-05 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489683+0000", 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.038e-06 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489735+0000", 2024-09-13T09:26:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1824999999999999e-05 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489826+0000", 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1533999999999995e-05 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.491305+0000", 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014785320000000001 2024-09-13T09:26:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.501618+0000", 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010312928000000001 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.506959+0000", 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053409310000000002 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.507065+0000", 2024-09-13T09:26:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105925 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.507145+0000", 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0455000000000001e-05 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2334 3.2 3:43c4f885:::rbd_data.1118e3e5e25d.000000000000019f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.649614+0000", 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.21581473, 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0061120749999999998, 2024-09-13T09:26:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2334 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649614+0000", 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649614+0000", 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649612+0000", 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649623+0000", 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0986e-05 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649640+0000", 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6441999999999999e-05 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649651+0000", 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0706999999999999e-05 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649775+0000", 2024-09-13T09:26:09.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000124735 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649891+0000", 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115851 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.650115+0000", 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223845 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.651399+0000", 2024-09-13T09:26:09.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012840779999999999 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.655671+0000", 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042715399999999999 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.655703+0000", 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2815999999999999e-05 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.655726+0000", 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2880000000000001e-05 2024-09-13T09:26:09.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2815 3.1e 3:7b894b3d:::rbd_data.11189888a71f.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.9817924759999999, 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02585463, 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2815 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883635+0000", 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883648+0000", 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3205e-05 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883682+0000", 2024-09-13T09:26:09.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4421000000000001e-05 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883700+0000", 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7501000000000001e-05 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883780+0000", 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0418000000000002e-05 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883885+0000", 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105368 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.889905+0000", 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0060196479999999998 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909412+0000", 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019506672999999999 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909424+0000", 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2692e-05 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909462+0000", 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7069e-05 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909491+0000", 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9560000000000002e-05 2024-09-13T09:26:09.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2922 3.7 3:e2b2cb88:::rbd_data.11189888a71f.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.119837+0000", 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7455915050000002, 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0073683689999999996, 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.958 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2922 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119837+0000", 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119837+0000", 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119836+0000", 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119846+0000", 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7690000000000008e-06 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119868+0000", 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2212999999999999e-05 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119879+0000", 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0892e-05 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119937+0000", 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8143999999999998e-05 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.120008+0000", 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1019000000000006e-05 2024-09-13T09:26:09.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.120806+0000", 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00079801899999999996 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.125034+0000", 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042281469999999998 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.127126+0000", 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020916649999999999 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.127173+0000", 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7521000000000001e-05 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.127206+0000", 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2503999999999999e-05 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2990 3.1e 3:7b894b3d:::rbd_data.11189888a71f.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.259708+0000", 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6057205870000004, 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012073198, 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2990 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259708+0000", 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259708+0000", 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259706+0000", 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259717+0000", 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0706999999999999e-05 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259738+0000", 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0661000000000001e-05 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259747+0000", 2024-09-13T09:26:09.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9600000000000006e-06 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259834+0000", 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7459000000000004e-05 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259985+0000", 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000150901 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.262254+0000", 2024-09-13T09:26:09.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022691740000000001 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271647+0000", 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093931459999999998 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271666+0000", 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8732e-05 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271727+0000", 2024-09-13T09:26:09.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0748000000000002e-05 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271782+0000", 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4768999999999997e-05 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3997 3.12 e35/25 3:49c74a32:::rbd_object_map.1118fb4e889d:head v 35'50, mlcod=34'48)", 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.449668+0000", 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.41576076000000001, 2024-09-13T09:26:09.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059057149999999997, 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449668+0000", 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449668+0000", 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449666+0000", 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449685+0000", 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9017000000000001e-05 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449713+0000", 2024-09-13T09:26:09.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8238999999999998e-05 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449943+0000", 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000229261 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.450185+0000", 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000241905 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.450254+0000", 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9565000000000002e-05 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.455537+0000", 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052828099999999998 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.455574+0000", 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6881000000000001e-05 2024-09-13T09:26:09.973 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4108 3.7 3:e0c663d0:::rbd_data.1118fb4e889d.0000000000000001:head [truncate 11010048] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.400912728, 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0082899110000000005, 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4108 2024-09-13T09:26:09.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464519+0000", 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4620000000000001e-06 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464532+0000", 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2656e-05 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464536+0000", 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465592+0000", 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001056076 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465636+0000", 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4790000000000003e-05 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465732+0000", 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5775999999999999e-05 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.471690+0000", 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059574600000000004 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472734+0000", 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001044747 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472776+0000", 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1371000000000001e-05 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472806+0000", 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0332999999999999e-05 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4111 3.b 3:d2e4b237:::rbd_data.1118fb4e889d.0000000000000005:head [truncate 10747904] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.465526+0000", 2024-09-13T09:26:09.979 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.39990312500000003, 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0072737959999999999, 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4111 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465526+0000", 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465526+0000", 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465525+0000", 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:09.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465528+0000", 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0810000000000001e-06 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465531+0000", 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.687e-06 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465534+0000", 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8150000000000002e-06 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466356+0000", 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082187700000000005 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466492+0000", 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013583099999999999 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466703+0000", 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000210912 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.471702+0000", 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004999206 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472734+0000", 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001032282 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472772+0000", 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8176000000000002e-05 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:09.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472800+0000", 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7282999999999999e-05 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:09.985 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:09.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:09.987 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:09.981+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:09.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:09.988 INFO:teuthology.orchestra.run.smithi063.stdout:578 punch from 0x47485b8 to 0x4755e23, (0xd86b bytes) 2024-09-13T09:26:09.988 INFO:teuthology.orchestra.run.smithi063.stdout:581 read 0xb71b0dc thru 0xb71be26 (0xd4b bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:582 writesame 0x94d9dc5 thru 0x94d9e2a data_size 0x3(0x66 bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:584 write 0x5c5c280 thru 0x5c6616a (0x9eeb bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:587 read 0xcd48a0b thru 0xcd55394 (0xc98a bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:588 punch from 0x8aa327d to 0x8ab1e03, (0xeb86 bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:590 punch from 0xab22bb to 0xab8e7a, (0x6bbf bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:592 write 0x9a86207 thru 0x9a95858 (0xf652 bytes) 2024-09-13T09:26:09.989 INFO:teuthology.orchestra.run.smithi063.stdout:593 writesame 0x53590a1 thru 0x5360dff data_size 0x31(0x7d5f bytes) 2024-09-13T09:26:09.990 INFO:teuthology.orchestra.run.smithi063.stdout:594 writesame 0x8fadb36 thru 0x8fb684c data_size 0x1(0x8d17 bytes) 2024-09-13T09:26:10.014 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.014 INFO:teuthology.orchestra.run.smithi063.stdout:596 writesame 0x537bb11 thru 0x538702c data_size 0x4(0xb51c bytes) 2024-09-13T09:26:10.020 INFO:teuthology.orchestra.run.smithi063.stdout:597 write 0x263e638 thru 0x263f6d9 (0x10a2 bytes) 2024-09-13T09:26:10.022 INFO:teuthology.orchestra.run.smithi063.stdout:598 read 0x385bf26 thru 0x385e38f (0x246a bytes) 2024-09-13T09:26:10.022 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.040 INFO:teuthology.orchestra.run.smithi063.stdout:600 trunc from 0xd993c9c to 0x3099e2b 2024-09-13T09:26:10.085 INFO:teuthology.orchestra.run.smithi063.stdout:601 write 0xce0954b thru 0xce0a0d3 (0xb89 bytes) 2024-09-13T09:26:10.087 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:10.082+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:10.088 INFO:teuthology.orchestra.run.smithi063.stdout:602 punch from 0xc0e2520 to 0xc0e82aa, (0x5d8a bytes) 2024-09-13T09:26:10.088 INFO:teuthology.orchestra.run.smithi063.stdout:604 write 0x697caa thru 0x6a3e4c (0xc1a3 bytes) 2024-09-13T09:26:10.102 INFO:teuthology.orchestra.run.smithi063.stdout:605 write 0xd01c75e thru 0xd02598d (0x9230 bytes) 2024-09-13T09:26:10.108 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:10.108 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:10.108 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:10.118 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:10.123 INFO:teuthology.orchestra.run.smithi063.stdout:606 trunc from 0xd02598e to 0x987cd6 2024-09-13T09:26:10.147 INFO:teuthology.orchestra.run.smithi063.stdout:607 read 0x9404cc thru 0x940c79 (0x7ae bytes) 2024-09-13T09:26:10.147 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.147 INFO:teuthology.orchestra.run.smithi063.stdout:609 read 0x3b2c97 thru 0x3bfb3a (0xcea4 bytes) 2024-09-13T09:26:10.148 INFO:teuthology.orchestra.run.smithi063.stdout:610 punch from 0x634957 to 0x643339, (0xe9e2 bytes) 2024-09-13T09:26:10.148 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.148 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.148 INFO:teuthology.orchestra.run.smithi063.stdout:614 punch from 0x6ca631 to 0x6cec00, (0x45cf bytes) 2024-09-13T09:26:10.149 INFO:teuthology.orchestra.run.smithi063.stdout:615 read 0x3f4b40 thru 0x3f96fb (0x4bbc bytes) 2024-09-13T09:26:10.149 INFO:teuthology.orchestra.run.smithi063.stdout:617 read 0x70b08c thru 0x712ac5 (0x7a3a bytes) 2024-09-13T09:26:10.159 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:10.164 INFO:teuthology.orchestra.run.smithi063.stdout:618 writesame 0x3c1dc04 thru 0x3c29555 data_size 0x3(0xb952 bytes) 2024-09-13T09:26:10.166 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.167 INFO:teuthology.orchestra.run.smithi063.stdout:621 read 0x272907f thru 0x272d86c (0x47ee bytes) 2024-09-13T09:26:10.167 INFO:teuthology.orchestra.run.smithi063.stdout:622 punch from 0x156865f to 0x15735a4, (0xaf45 bytes) 2024-09-13T09:26:10.181 INFO:teuthology.orchestra.run.smithi063.stdout:623 writesame 0x5f8a1cd thru 0x5f99d27 data_size 0x3(0xfb5b bytes) 2024-09-13T09:26:10.187 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:10.188 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:10.182+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:10.189 INFO:teuthology.orchestra.run.smithi063.stdout:624 trunc from 0x5f99d28 to 0x3481eed 2024-09-13T09:26:10.206 INFO:teuthology.orchestra.run.smithi063.stdout:625 trunc from 0x3481eed to 0x55b755e 2024-09-13T09:26:10.215 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:10.225 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:10.225 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:10.225 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:10.233 INFO:teuthology.orchestra.run.smithi063.stdout:626 write 0x7b80400 thru 0x7b83b57 (0x3758 bytes) 2024-09-13T09:26:10.235 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:10.236 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:10.239 INFO:teuthology.orchestra.run.smithi063.stdout:627 punch from 0x1b34c1e to 0x1b4067d, (0xba5f bytes) 2024-09-13T09:26:10.240 INFO:teuthology.orchestra.run.smithi063.stdout:628 trunc from 0x7b83b58 to 0x773de2c 2024-09-13T09:26:10.260 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:10.270 INFO:teuthology.orchestra.run.smithi063.stdout:629 write 0xb402ccf thru 0xb405220 (0x2552 bytes) 2024-09-13T09:26:10.271 INFO:teuthology.orchestra.run.smithi063.stdout:630 read 0x987740a thru 0x9880ad4 (0x96cb bytes) 2024-09-13T09:26:10.272 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.272 INFO:teuthology.orchestra.run.smithi063.stdout:634 write 0xdaad02 thru 0xdafab1 (0x4db0 bytes) 2024-09-13T09:26:10.278 INFO:teuthology.orchestra.run.smithi063.stdout:635 read 0x22cbdcd thru 0x22d9fda (0xe20e bytes) 2024-09-13T09:26:10.282 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:10.289 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:10.285+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:10.294 INFO:teuthology.orchestra.run.smithi063.stdout:636 write 0xe179de5 thru 0xe1839d6 (0x9bf2 bytes) 2024-09-13T09:26:10.297 INFO:teuthology.orchestra.run.smithi063.stdout:638 write 0x5526230 thru 0x552674f (0x520 bytes) 2024-09-13T09:26:10.306 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:10.308 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:26:10.314 INFO:teuthology.orchestra.run.smithi063.stdout:639 trunc from 0xe1839d7 to 0x848eb04 2024-09-13T09:26:10.329 INFO:teuthology.orchestra.run.smithi063.stdout:641 read 0x834b1d thru 0x836e48 (0x232c bytes) 2024-09-13T09:26:10.335 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:10.337 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:10.337 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:10.337 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:10.337 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:10.337 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:10.338 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:10.338 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:10.338 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:10.340 INFO:teuthology.orchestra.run.smithi063.stdout:642 trunc from 0x848eb04 to 0x1f5102e 2024-09-13T09:26:10.342 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:10.342 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:10.352 INFO:teuthology.orchestra.run.smithi063.stdout:643 read 0x1941988 thru 0x1944fbb (0x3634 bytes) 2024-09-13T09:26:10.353 INFO:teuthology.orchestra.run.smithi063.stdout:644 read 0x19983c5 thru 0x19a40cd (0xbd09 bytes) 2024-09-13T09:26:10.353 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.353 INFO:teuthology.orchestra.run.smithi063.stdout:646 punch from 0x1c757f0 to 0x1c81945, (0xc155 bytes) 2024-09-13T09:26:10.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:10.383 INFO:teuthology.orchestra.run.smithi063.stdout:647 writesame 0xce3b551 thru 0xce40b1b data_size 0x1(0x55cb bytes) 2024-09-13T09:26:10.385 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.390 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:10.382+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:10.397 INFO:teuthology.orchestra.run.smithi063.stdout:650 write 0xd6ce7fa thru 0xd6d8d2b (0xa532 bytes) 2024-09-13T09:26:10.411 INFO:teuthology.orchestra.run.smithi063.stdout:652 trunc from 0xd6d8d2c to 0x9a06a30 2024-09-13T09:26:10.448 INFO:teuthology.orchestra.run.smithi063.stdout:653 writesame 0xdcc23a0 thru 0xdcd087b data_size 0x61(0xe4dc bytes) 2024-09-13T09:26:10.450 INFO:teuthology.orchestra.run.smithi063.stdout:654 write 0x3552a1c thru 0x355c3dd (0x99c2 bytes) 2024-09-13T09:26:10.453 INFO:teuthology.orchestra.run.smithi063.stdout:655 writesame 0xf08291 thru 0xf105ef data_size 0xa9(0x835f bytes) 2024-09-13T09:26:10.456 INFO:teuthology.orchestra.run.smithi063.stdout:656 punch from 0xc3e8a37 to 0xc3ee71e, (0x5ce7 bytes) 2024-09-13T09:26:10.456 INFO:teuthology.orchestra.run.smithi063.stdout:657 read 0x13b027e thru 0x13b06c8 (0x44b bytes) 2024-09-13T09:26:10.461 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:10.461 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:10.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:10.480 INFO:teuthology.orchestra.run.smithi063.stdout:658 trunc from 0xdcd087c to 0x332e4f6 2024-09-13T09:26:10.491 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:10.486+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:10.494 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.510 INFO:teuthology.orchestra.run.smithi063.stdout:661 writesame 0x4e222e8 thru 0x4e27b22 data_size 0x1(0x583b bytes) 2024-09-13T09:26:10.512 INFO:teuthology.orchestra.run.smithi063.stdout:662 read 0x30c23d3 thru 0x30d060b (0xe239 bytes) 2024-09-13T09:26:10.531 INFO:teuthology.orchestra.run.smithi063.stdout:663 trunc from 0x4e27b23 to 0xd533c95 2024-09-13T09:26:10.543 INFO:teuthology.orchestra.run.smithi063.stdout:664 write 0x2aa6f7a thru 0x2ab0fa7 (0xa02e bytes) 2024-09-13T09:26:10.546 INFO:teuthology.orchestra.run.smithi063.stdout:665 punch from 0x11c1e23 to 0x11ca9e4, (0x8bc1 bytes) 2024-09-13T09:26:10.553 INFO:teuthology.orchestra.run.smithi063.stdout:666 trunc from 0xd533c95 to 0xa4d065f 2024-09-13T09:26:10.566 INFO:teuthology.orchestra.run.smithi063.stdout:667 read 0x3f4f8f5 thru 0x3f5b1a4 (0xb8b0 bytes) 2024-09-13T09:26:10.567 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.567 INFO:teuthology.orchestra.run.smithi063.stdout:669 punch from 0x9008414 to 0x9011d0f, (0x98fb bytes) 2024-09-13T09:26:10.567 INFO:teuthology.orchestra.run.smithi063.stdout:671 read 0x64f91a7 thru 0x6504c78 (0xbad2 bytes) 2024-09-13T09:26:10.567 INFO:teuthology.orchestra.run.smithi063.stdout:674 punch from 0x324b53f to 0x32571da, (0xbc9b bytes) 2024-09-13T09:26:10.584 INFO:teuthology.orchestra.run.smithi063.stdout:675 writesame 0xd60fc98 thru 0xd614127 data_size 0x8(0x4490 bytes) 2024-09-13T09:26:10.585 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:10.585 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:10.587 INFO:teuthology.orchestra.run.smithi063.stdout:676 write 0xcb87aa5 thru 0xcb8a7ce (0x2d2a bytes) 2024-09-13T09:26:10.590 INFO:teuthology.orchestra.run.smithi063.stdout:677 read 0x2d67db4 thru 0x2d74069 (0xc2b6 bytes) 2024-09-13T09:26:10.591 INFO:teuthology.orchestra.run.smithi063.stdout:678 read 0xa933750 thru 0xa94043d (0xccee bytes) 2024-09-13T09:26:10.591 INFO:teuthology.orchestra.run.smithi063.stdout:679 writesame 0x682dab7 thru 0x6831d65 data_size 0x1(0x42af bytes) 2024-09-13T09:26:10.592 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:10.586+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:10.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:10.626 INFO:teuthology.orchestra.run.smithi063.stdout:680 trunc from 0xd614128 to 0x2f01d9b 2024-09-13T09:26:10.675 INFO:teuthology.orchestra.run.smithi063.stdout:683 write 0xea91cee thru 0xea94d02 (0x3015 bytes) 2024-09-13T09:26:10.678 INFO:teuthology.orchestra.run.smithi063.stdout:684 read 0x986d633 thru 0x9878068 (0xaa36 bytes) 2024-09-13T09:26:10.679 INFO:teuthology.orchestra.run.smithi063.stdout:685 read 0x8489265 thru 0x84964d7 (0xd273 bytes) 2024-09-13T09:26:10.680 INFO:teuthology.orchestra.run.smithi063.stdout:688 read 0xe38f96f thru 0xe391c84 (0x2316 bytes) 2024-09-13T09:26:10.681 INFO:teuthology.orchestra.run.smithi063.stdout:689 read 0x5e41a96 thru 0x5e4380a (0x1d75 bytes) 2024-09-13T09:26:10.681 INFO:teuthology.orchestra.run.smithi063.stdout:690 punch from 0x6d20ea3 to 0x6d223eb, (0x1548 bytes) 2024-09-13T09:26:10.681 INFO:teuthology.orchestra.run.smithi063.stdout:694 writesame 0x981f577 thru 0x9821e37 data_size 0x1(0x28c1 bytes) 2024-09-13T09:26:10.690 INFO:teuthology.orchestra.run.smithi063.stdout:695 write 0x1f35863 thru 0x1f3fe50 (0xa5ee bytes) 2024-09-13T09:26:10.692 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:10.686+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:10.694 INFO:teuthology.orchestra.run.smithi063.stdout:696 punch from 0xea94df to 0xeb10c5, (0x7be6 bytes) 2024-09-13T09:26:10.694 INFO:teuthology.orchestra.run.smithi063.stdout:698 write 0xbc85ab6 thru 0xbc95009 (0xf554 bytes) 2024-09-13T09:26:10.697 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:10.697 INFO:teuthology.orchestra.run.smithi063.stdout:700 write 0x7fe6c5e thru 0x7fef69b (0x8a3e bytes) 2024-09-13T09:26:10.702 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:10.702 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:10.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:10.729 INFO:teuthology.orchestra.run.smithi063.stdout:701 trunc from 0xea94d03 to 0xdafe42 2024-09-13T09:26:10.779 INFO:teuthology.orchestra.run.smithi063.stdout:702 trunc from 0xdafe42 to 0xe81143c 2024-09-13T09:26:10.791 INFO:teuthology.orchestra.run.smithi063.stdout:705 clone 8 order 24 su 1048576 sc 7 2024-09-13T09:26:10.793 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:10.786+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:10.817 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:10.817 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:10.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:10.894 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:10.890+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:10.929 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:10.929 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:10.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:10.995 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:10.990+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:11.044 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.044 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:11.097 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:11.090+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:11.160 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.160 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.170 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:11.197 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:11.190+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:11.273 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.273 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:11.298 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:11.293+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:11.394 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.394 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.399 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:11.393+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:11.404 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:11.500 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:11.493+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:11.517 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone7 from 0xe81143c (overlap 0x987cd6) to 0x4c3e6b 2024-09-13T09:26:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:11.602 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:11.597+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:11.638 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.638 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.649 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:11.702 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:11.698+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:11.719 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T09:26:11.762 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.762 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:11.804 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:11.801+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:11.891 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:11.891 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:11.901 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:11.904 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:11.898+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:11.972 INFO:teuthology.orchestra.run.smithi063.stdout:707 punch from 0x52f7de1 to 0x5300e0c, (0x902b bytes) 2024-09-13T09:26:11.980 INFO:teuthology.orchestra.run.smithi063.stdout:708 write 0x8dcb68f thru 0x8dcf4eb (0x3e5d bytes) 2024-09-13T09:26:11.988 INFO:teuthology.orchestra.run.smithi063.stdout:709 write 0x8548086 thru 0x8554ded (0xcd68 bytes) 2024-09-13T09:26:11.996 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:11.996 INFO:teuthology.orchestra.run.smithi063.stdout:712 writesame 0xba03a48 thru 0xba1058f data_size 0x14(0xcb48 bytes) 2024-09-13T09:26:12.005 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:11.998+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:12.013 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:12.013 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:12.025 INFO:teuthology.orchestra.run.smithi063.stdout:713 trunc from 0xe81143c to 0x1db2246 2024-09-13T09:26:12.026 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:12.057 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.057 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.057 INFO:teuthology.orchestra.run.smithi063.stdout:716 read 0x1d372cd thru 0x1d3d1ce (0x5f02 bytes) 2024-09-13T09:26:12.057 INFO:teuthology.orchestra.run.smithi063.stdout:717 read 0x1288820 thru 0x1288862 (0x43 bytes) 2024-09-13T09:26:12.058 INFO:teuthology.orchestra.run.smithi063.stdout:719 read 0xda3ff1 thru 0xda447e (0x48e bytes) 2024-09-13T09:26:12.079 INFO:teuthology.orchestra.run.smithi063.stdout:721 writesame 0x93a742a thru 0x93ae8e6 data_size 0x2b(0x74bd bytes) 2024-09-13T09:26:12.080 INFO:teuthology.orchestra.run.smithi063.stdout:722 read 0x1092178 thru 0x1093e4c (0x1cd5 bytes) 2024-09-13T09:26:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:724 writesame 0xb89fa42 thru 0xb89ff46 data_size 0x1(0x505 bytes) 2024-09-13T09:26:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:725 read 0x381c994 thru 0x3821e36 (0x54a3 bytes) 2024-09-13T09:26:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:728 write 0x78b39ec thru 0x78bede8 (0xb3fd bytes) 2024-09-13T09:26:12.105 INFO:teuthology.orchestra.run.smithi063.stdout:729 trunc from 0xb89ff47 to 0xdd72338 2024-09-13T09:26:12.106 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:12.098+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:12.115 INFO:teuthology.orchestra.run.smithi063.stdout:730 writesame 0xa90d9e thru 0xa92aee data_size 0x13(0x1d51 bytes) 2024-09-13T09:26:12.117 INFO:teuthology.orchestra.run.smithi063.stdout:732 read 0xcf81d36 thru 0xcf87fed (0x62b8 bytes) 2024-09-13T09:26:12.117 INFO:teuthology.orchestra.run.smithi063.stdout:733 write 0x35e8e6d thru 0x35f2a76 (0x9c0a bytes) 2024-09-13T09:26:12.133 INFO:teuthology.orchestra.run.smithi063.stdout:735 write 0xe5a24e0 thru 0xe5a270b (0x22c bytes) 2024-09-13T09:26:12.137 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:12.137 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:12.139 INFO:teuthology.orchestra.run.smithi063.stdout:737 punch from 0x44eaadc to 0x44f6ebc, (0xc3e0 bytes) 2024-09-13T09:26:12.139 INFO:teuthology.orchestra.run.smithi063.stdout:738 punch from 0xb8f1e1f to 0xb8f31e7, (0x13c8 bytes) 2024-09-13T09:26:12.139 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.140 INFO:teuthology.orchestra.run.smithi063.stdout:741 write 0xe19719a thru 0xe19d001 (0x5e68 bytes) 2024-09-13T09:26:12.145 INFO:teuthology.orchestra.run.smithi063.stdout:742 read 0x3d53b0e thru 0x3d5ddb4 (0xa2a7 bytes) 2024-09-13T09:26:12.146 INFO:teuthology.orchestra.run.smithi063.stdout:744 writesame 0xa59bf6b thru 0xa5a3964 data_size 0xd(0x79fa bytes) 2024-09-13T09:26:12.148 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:12.154 INFO:teuthology.orchestra.run.smithi063.stdout:745 writesame 0x17b0cef thru 0x17b0e1b data_size 0x1(0x12d bytes) 2024-09-13T09:26:12.176 INFO:teuthology.orchestra.run.smithi063.stdout:746 trunc from 0xe5a270c to 0x3df1b99 2024-09-13T09:26:12.207 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:12.202+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:747 writesame 0xcd34ad3 thru 0xcd39e94 data_size 0x2(0x53c2 bytes) 2024-09-13T09:26:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:749 write 0x640953c thru 0x640cf46 (0x3a0b bytes) 2024-09-13T09:26:12.229 INFO:teuthology.orchestra.run.smithi063.stdout:750 read 0x9b22e07 thru 0x9b26722 (0x391c bytes) 2024-09-13T09:26:12.229 INFO:teuthology.orchestra.run.smithi063.stdout:751 read 0x438fed8 thru 0x43967f2 (0x691b bytes) 2024-09-13T09:26:12.229 INFO:teuthology.orchestra.run.smithi063.stdout:752 read 0x99e0790 thru 0x99e8799 (0x800a bytes) 2024-09-13T09:26:12.230 INFO:teuthology.orchestra.run.smithi063.stdout:753 punch from 0x4fc99b3 to 0x4fd2f35, (0x9582 bytes) 2024-09-13T09:26:12.230 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.230 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.230 INFO:teuthology.orchestra.run.smithi063.stdout:757 read 0x7da60a9 thru 0x7daff95 (0x9eed bytes) 2024-09-13T09:26:12.230 INFO:teuthology.orchestra.run.smithi063.stdout:758 write 0xb8dedd thru 0xb9220e (0x4332 bytes) 2024-09-13T09:26:12.232 INFO:teuthology.orchestra.run.smithi063.stdout:759 write 0x67bfd64 thru 0x67cec68 (0xef05 bytes) 2024-09-13T09:26:12.249 INFO:teuthology.orchestra.run.smithi063.stdout:760 write 0xd2ec1e2 thru 0xd2f4f24 (0x8d43 bytes) 2024-09-13T09:26:12.255 INFO:teuthology.orchestra.run.smithi063.stdout:762 write 0xaa336c5 thru 0xaa368c7 (0x3203 bytes) 2024-09-13T09:26:12.261 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.261 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.261 INFO:teuthology.orchestra.run.smithi063.stdout:765 write 0xad76536 thru 0xad7cd3d (0x6808 bytes) 2024-09-13T09:26:12.270 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:12.270 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:12.270 INFO:teuthology.orchestra.run.smithi063.stdout:766 trunc from 0xd2f4f25 to 0xeaf6420 2024-09-13T09:26:12.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:12.282 INFO:teuthology.orchestra.run.smithi063.stdout:767 write 0x88e5f0 thru 0x8928a8 (0x42b9 bytes) 2024-09-13T09:26:12.285 INFO:teuthology.orchestra.run.smithi063.stdout:769 punch from 0xc487d6 to 0xc53108, (0xa932 bytes) 2024-09-13T09:26:12.285 INFO:teuthology.orchestra.run.smithi063.stdout:770 write 0xac9d7e2 thru 0xaca705f (0x987e bytes) 2024-09-13T09:26:12.292 INFO:teuthology.orchestra.run.smithi063.stdout:773 punch from 0x1e9c619 to 0x1ea09f7, (0x43de bytes) 2024-09-13T09:26:12.292 INFO:teuthology.orchestra.run.smithi063.stdout:774 writesame 0xacec210 thru 0xacf7107 data_size 0x2c(0xaef8 bytes) 2024-09-13T09:26:12.296 INFO:teuthology.orchestra.run.smithi063.stdout:775 write 0xd86d252 thru 0xd87cad4 (0xf883 bytes) 2024-09-13T09:26:12.303 INFO:teuthology.orchestra.run.smithi063.stdout:776 write 0x59700c6 thru 0x5979940 (0x987b bytes) 2024-09-13T09:26:12.306 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.307 INFO:teuthology.orchestra.run.smithi063.stdout:778 trunc from 0xeaf6420 to 0xe0928f8 2024-09-13T09:26:12.308 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:12.301+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:12.320 INFO:teuthology.orchestra.run.smithi063.stdout:779 write 0x4606ae4 thru 0x46099e1 (0x2efe bytes) 2024-09-13T09:26:12.328 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:26:12.345 INFO:teuthology.orchestra.run.smithi063.stdout:781 trunc from 0xe0928f8 to 0x228fdc4 2024-09-13T09:26:12.350 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:12.373 INFO:teuthology.orchestra.run.smithi063.stdout:782 read 0x499477 thru 0x49ae7d (0x1a07 bytes) 2024-09-13T09:26:12.373 INFO:teuthology.orchestra.run.smithi063.stdout:784 read 0x175869c thru 0x175b3d4 (0x2d39 bytes) 2024-09-13T09:26:12.374 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.374 INFO:teuthology.orchestra.run.smithi063.stdout:786 read 0x6f5c44 thru 0x6f5f15 (0x2d2 bytes) 2024-09-13T09:26:12.377 INFO:teuthology.orchestra.run.smithi063.stdout:788 trunc from 0x228fdc4 to 0xb29f36 2024-09-13T09:26:12.399 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:12.400 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:12.409 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:12.405+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:12.413 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:12.418 INFO:teuthology.orchestra.run.smithi063.stdout:789 write 0x8d47e06 thru 0x8d53019 (0xb214 bytes) 2024-09-13T09:26:12.438 INFO:teuthology.orchestra.run.smithi063.stdout:790 writesame 0xb0ecb95 thru 0xb0f5250 data_size 0x2(0x86bc bytes) 2024-09-13T09:26:12.445 INFO:teuthology.orchestra.run.smithi063.stdout:791 trunc from 0xb0f5251 to 0xdddbdfa 2024-09-13T09:26:12.457 INFO:teuthology.orchestra.run.smithi063.stdout:792 writesame 0x51d96ab thru 0x51e042c data_size 0x2(0x6d82 bytes) 2024-09-13T09:26:12.458 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:26:12.459 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:26:12.470 INFO:teuthology.orchestra.run.smithi063.stdout:793 write 0x84a1476 thru 0x84abadd (0xa668 bytes) 2024-09-13T09:26:12.476 INFO:teuthology.orchestra.run.smithi063.stdout:795 writesame 0x4e6bf39 thru 0x4e75f7a data_size 0x49(0xa042 bytes) 2024-09-13T09:26:12.479 INFO:teuthology.orchestra.run.smithi063.stdout:796 read 0x90327ef thru 0x9038446 (0x5c58 bytes) 2024-09-13T09:26:12.479 INFO:teuthology.orchestra.run.smithi063.stdout:797 read 0x71cfd1c thru 0x71d2679 (0x295e bytes) 2024-09-13T09:26:12.480 INFO:teuthology.orchestra.run.smithi063.stdout:798 write 0xa217210 thru 0xa2258db (0xe6cc bytes) 2024-09-13T09:26:12.483 INFO:teuthology.orchestra.run.smithi063.stdout:799 writesame 0xba88d97 thru 0xba8929f data_size 0x1(0x509 bytes) 2024-09-13T09:26:12.489 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.505 INFO:teuthology.orchestra.run.smithi063.stdout:802 trunc from 0xdddbdfa to 0x428c136 2024-09-13T09:26:12.510 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:12.502+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:12.525 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:12.528 INFO:teuthology.orchestra.run.smithi063.stdout:808 read 0x23e085a thru 0x23eeead (0xe654 bytes) 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:809 writesame 0xcb35f1b thru 0xcb3f788 data_size 0x2(0x986e bytes) 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:813 read 0xb6be5b6 thru 0xb6cab13 (0xc55e bytes) 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:817 punch from 0x6fdd8ca to 0x6fe2553, (0x4c89 bytes) 2024-09-13T09:26:12.554 INFO:teuthology.orchestra.run.smithi063.stdout:818 punch from 0x3daf70 to 0x3de784, (0x3814 bytes) 2024-09-13T09:26:12.555 INFO:teuthology.orchestra.run.smithi063.stdout:819 write 0x60b3f3c thru 0x60bb9d8 (0x7a9d bytes) 2024-09-13T09:26:12.555 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:12.555 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:820 read 0xc4f95b5 thru 0xc5046a3 (0xb0ef bytes) 2024-09-13T09:26:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:821 punch from 0xc9ad3e4 to 0xc9b62c1, (0x8edd bytes) 2024-09-13T09:26:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:822 punch from 0x5360dbd to 0x536f848, (0xea8b bytes) 2024-09-13T09:26:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:823 read 0x76d6584 thru 0x76e28ec (0xc369 bytes) 2024-09-13T09:26:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:824 write 0xc082d21 thru 0xc08f043 (0xc323 bytes) 2024-09-13T09:26:12.560 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:12.568 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:12.576 INFO:teuthology.orchestra.run.smithi063.stdout:826 writesame 0xdf0e59b thru 0xdf177c7 data_size 0x1(0x922d bytes) 2024-09-13T09:26:12.576 INFO:teuthology.orchestra.run.smithi063.stdout:827 read 0xc92b239 thru 0xc92bee4 (0xcac bytes) 2024-09-13T09:26:12.577 INFO:teuthology.orchestra.run.smithi063.stdout:828 write 0x7022f1f thru 0x7032c87 (0xfd69 bytes) 2024-09-13T09:26:12.585 INFO:teuthology.orchestra.run.smithi063.stdout:829 trunc from 0xdf177c8 to 0xd064c08 2024-09-13T09:26:12.590 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:12.597 INFO:teuthology.orchestra.run.smithi063.stdout:830 punch from 0x4d1001c to 0x4d1a357, (0xa33b bytes) 2024-09-13T09:26:12.598 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.598 INFO:teuthology.orchestra.run.smithi063.stdout:832 read 0xb3decd5 thru 0xb3ea287 (0xb5b3 bytes) 2024-09-13T09:26:12.598 INFO:teuthology.orchestra.run.smithi063.stdout:833 write 0xfb7ae0 thru 0xfc2278 (0xa799 bytes) 2024-09-13T09:26:12.600 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.601 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.601 INFO:teuthology.orchestra.run.smithi063.stdout:836 read 0xa0b2486 thru 0xa0bb5a8 (0x9123 bytes) 2024-09-13T09:26:12.602 INFO:teuthology.orchestra.run.smithi063.stdout:837 trunc from 0xd064c08 to 0xdc732dc 2024-09-13T09:26:12.611 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:12.605+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:12.613 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:12.620 INFO:teuthology.orchestra.run.smithi063.stdout:838 trunc from 0xdc732dc to 0x8730f87 2024-09-13T09:26:12.633 INFO:teuthology.orchestra.run.smithi063.stdout:839 read 0x5a8ca23 thru 0x5a9322f (0x680d bytes) 2024-09-13T09:26:12.634 INFO:teuthology.orchestra.run.smithi063.stdout:840 punch from 0x6cfcb2c to 0x6d0a172, (0xd646 bytes) 2024-09-13T09:26:12.634 INFO:teuthology.orchestra.run.smithi063.stdout:843 punch from 0x4db5ba9 to 0x4db8a9e, (0x2ef5 bytes) 2024-09-13T09:26:12.634 INFO:teuthology.orchestra.run.smithi063.stdout:845 write 0x2852105 thru 0x2852b84 (0xa80 bytes) 2024-09-13T09:26:12.636 INFO:teuthology.orchestra.run.smithi063.stdout:846 read 0x5775ea7 thru 0x577af00 (0x505a bytes) 2024-09-13T09:26:12.642 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:12.647 INFO:teuthology.orchestra.run.smithi063.stdout:847 trunc from 0x8730f87 to 0x29178c0 2024-09-13T09:26:12.677 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:848 writesame 0x83125de thru 0x8315c66 data_size 0x1(0x3689 bytes) 2024-09-13T09:26:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:849 read 0x5cfe5a3 thru 0x5d09363 (0xadc1 bytes) 2024-09-13T09:26:12.688 INFO:teuthology.orchestra.run.smithi063.stdout:850 read 0xf1118f thru 0xf17c5c (0x6ace bytes) 2024-09-13T09:26:12.689 INFO:teuthology.orchestra.run.smithi063.stdout:851 write 0x51c6908 thru 0x51d13b7 (0xaab0 bytes) 2024-09-13T09:26:12.692 INFO:teuthology.orchestra.run.smithi063.stdout:852 punch from 0x107e939 to 0x10822c0, (0x3987 bytes) 2024-09-13T09:26:12.693 INFO:teuthology.orchestra.run.smithi063.stdout:853 writesame 0x3e80ee9 thru 0x3e90535 data_size 0x1(0xf64d bytes) 2024-09-13T09:26:12.700 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:12.712 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:12.706+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:12.721 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:12.721 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:12.722 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:12.723 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:12.723 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:12.723 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:12.724 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:12.724 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:12.724 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:12.724 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:12.724 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:12.732 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:12.760 INFO:teuthology.orchestra.run.smithi063.stdout:855 write 0xe454f54 thru 0xe45689d (0x194a bytes) 2024-09-13T09:26:12.782 INFO:teuthology.orchestra.run.smithi063.stdout:856 trunc from 0xe45689e to 0x5a1fe38 2024-09-13T09:26:12.813 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:12.809+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:12.826 INFO:teuthology.orchestra.run.smithi063.stdout:858 write 0xb03a668 thru 0xb046c84 (0xc61d bytes) 2024-09-13T09:26:12.832 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.848 INFO:teuthology.orchestra.run.smithi063.stdout:862 write 0xdbbfd04 thru 0xdbcc02d (0xc32a bytes) 2024-09-13T09:26:12.854 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:12.854 INFO:teuthology.orchestra.run.smithi063.stdout:864 read 0x7d82547 thru 0x7d8c952 (0xa40c bytes) 2024-09-13T09:26:12.854 INFO:teuthology.orchestra.run.smithi063.stdout:865 read 0x582be2 thru 0x589b98 (0x6fb7 bytes) 2024-09-13T09:26:12.869 INFO:teuthology.orchestra.run.smithi063.stdout:866 write 0xe3eca12 thru 0xe3fc63e (0xfc2d bytes) 2024-09-13T09:26:12.875 INFO:teuthology.orchestra.run.smithi063.stdout:867 read 0x4fc32d5 thru 0x4fc5336 (0x2062 bytes) 2024-09-13T09:26:12.876 INFO:teuthology.orchestra.run.smithi063.stdout:868 read 0x4ccf66 thru 0x4d84f3 (0xb58e bytes) 2024-09-13T09:26:12.877 INFO:teuthology.orchestra.run.smithi063.stdout:869 write 0x4c41bc7 thru 0x4c4a368 (0x87a2 bytes) 2024-09-13T09:26:12.891 INFO:teuthology.orchestra.run.smithi063.stdout:870 trunc from 0xe3fc63f to 0x7725414 2024-09-13T09:26:12.895 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:12.895 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:12.895 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:12.895 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:12.896 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:12.906 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:12.915 INFO:teuthology.orchestra.run.smithi063.stdout:871 write 0x1ed43c1 thru 0x1eda984 (0x65c4 bytes) 2024-09-13T09:26:12.915 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:12.910+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:12.918 INFO:teuthology.orchestra.run.smithi063.stdout:875 read 0x78f9ac thru 0x7908a6 (0xefb bytes) 2024-09-13T09:26:12.921 INFO:teuthology.orchestra.run.smithi063.stdout:876 trunc from 0x7725414 to 0x6727f95 2024-09-13T09:26:12.946 INFO:teuthology.orchestra.run.smithi063.stdout:877 writesame 0x82493f thru 0x82cdd3 data_size 0x1(0x8495 bytes) 2024-09-13T09:26:12.969 INFO:teuthology.orchestra.run.smithi063.stdout:878 clone 9 order 22 su 262144 sc 6 2024-09-13T09:26:13.015 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:13.013+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:13.020 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:12 3.18 3:1a426f9e:::rbd_header.11182e510213:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.811157243, 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0064571740000000004, 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 12 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204336+0000", 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204334+0000", 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204345+0000", 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0258000000000001e-05 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204358+0000", 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3867e-05 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204372+0000", 2024-09-13T09:26:13.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3754000000000001e-05 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204436+0000", 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.204440+0000", 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207020+0000", 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025802910000000002 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207135+0000", 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114299 2024-09-13T09:26:13.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207374+0000", 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023949400000000001 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208859+0000", 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014852209999999999 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210662+0000", 2024-09-13T09:26:13.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001802894 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210738+0000", 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5448e-05 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.210793+0000", 2024-09-13T09:26:13.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5628999999999999e-05 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:274 3.7 3:e1d7b6d3:::rbd_data.111867511a81.0000000000000024:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,truncate 786432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.501841+0000", 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.513651979, 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062127679999999996, 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.029 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 274 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501841+0000", 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501841+0000", 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501840+0000", 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501852+0000", 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2483000000000001e-05 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501872+0000", 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0095999999999999e-05 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.501894+0000", 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1716e-05 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.503657+0000", 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001762668 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504024+0000", 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036737199999999998 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504720+0000", 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069563999999999997 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.506128+0000", 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014082789999999999 2024-09-13T09:26:13.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507873+0000", 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017447719999999999 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507969+0000", 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6551000000000004e-05 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.508054+0000", 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4857000000000004e-05 2024-09-13T09:26:13.035 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:421 3.8 3:11e50e73:::rbd_data.111867511a81.000000000000007b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.880682+0000", 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.13481125, 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056711770000000003, 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.036 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 421 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880682+0000", 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880682+0000", 2024-09-13T09:26:13.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880680+0000", 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880694+0000", 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3807e-05 2024-09-13T09:26:13.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880726+0000", 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1628999999999998e-05 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880740+0000", 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.880826+0000", 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6417999999999998e-05 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.881013+0000", 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000187269 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.881727+0000", 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071376000000000002 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.885596+0000", 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038691670000000002 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.886198+0000", 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00060181399999999999 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.886286+0000", 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7887999999999996e-05 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.886353+0000", 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7402000000000003e-05 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.042 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:448 3.8 3:16185ba7:::rbd_data.111867511a81.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.062666596, 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022114892000000001, 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.043 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 448 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952824+0000", 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:13.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952840+0000", 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5420000000000001e-05 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952875+0000", 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5076000000000001e-05 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952890+0000", 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5596000000000001e-05 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.953033+0000", 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014238400000000001 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.953282+0000", 2024-09-13T09:26:13.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024892700000000002 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.959632+0000", 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.006350372 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974849+0000", 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015217229000000001 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974877+0000", 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7421999999999999e-05 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974914+0000", 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7320000000000002e-05 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974942+0000", 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7763000000000002e-05 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:524 3.1e 3:7c1f840a:::rbd_data.111867511a81.00000000000000d0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:00.164928+0000", 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.850565609, 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063352649999999996, 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 524 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.050 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164928+0000", 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164928+0000", 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164925+0000", 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:13.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164943+0000", 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8182e-05 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164982+0000", 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8271e-05 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.164998+0000", 2024-09-13T09:26:13.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6639000000000001e-05 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.165093+0000", 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5209000000000003e-05 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.165101+0000", 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0700000000000007e-06 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.169355+0000", 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042532380000000003 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.169531+0000", 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017661399999999999 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.169748+0000", 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021657999999999999 2024-09-13T09:26:13.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.170613+0000", 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00086520100000000001 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.171135+0000", 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00052221899999999998 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.171183+0000", 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7420999999999999e-05 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:00.171263+0000", 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0317999999999999e-05 2024-09-13T09:26:13.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:1798 3.8 3:1362e7cf:::rbd_data.1118e3e5e25d.00000000000002ae:head [set-alloc-hint object_size 262144 write_size 262144,write 244159~17985] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:02.997512+0000", 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.017981832, 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059031409999999998, 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 1798 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997512+0000", 2024-09-13T09:26:13.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997512+0000", 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997511+0000", 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997523+0000", 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2473000000000001e-05 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997548+0000", 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5377000000000001e-05 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997554+0000", 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9340000000000003e-06 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997595+0000", 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0147e-05 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997672+0000", 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7785000000000001e-05 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:02.997757+0000", 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4976999999999996e-05 2024-09-13T09:26:13.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.000666+0000", 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029086450000000001 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.003287+0000", 2024-09-13T09:26:13.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026208550000000001 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.003341+0000", 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4293999999999999e-05 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.003415+0000", 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3578000000000006e-05 2024-09-13T09:26:13.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2047 3.b 3:d68ef2c4:::rbd_data.1118e3e5e25d.000000000000001b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.202051+0000", 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.8134424609999993, 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056841720000000004, 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2047 2024-09-13T09:26:13.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.066 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202051+0000", 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202051+0000", 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202049+0000", 2024-09-13T09:26:13.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202063+0000", 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4114000000000001e-05 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202103+0000", 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0541999999999999e-05 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202122+0000", 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8499e-05 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202262+0000", 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014035100000000001 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202419+0000", 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015647400000000001 2024-09-13T09:26:13.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202549+0000", 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013001399999999999 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.203917+0000", 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013679639999999999 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207624+0000", 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037072659999999999 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207679+0000", 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4552000000000002e-05 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207735+0000", 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6541e-05 2024-09-13T09:26:13.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2048 3.8 3:119bd4bc:::rbd_data.1118e3e5e25d.000000000000011f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.202159+0000", 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.8133343830000008, 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005693048, 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2048 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202159+0000", 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202159+0000", 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202158+0000", 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202167+0000", 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.363e-06 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202180+0000", 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.272e-05 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202188+0000", 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4849999999999992e-06 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202257+0000", 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9160999999999997e-05 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202364+0000", 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010705300000000001 2024-09-13T09:26:13.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.202486+0000", 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012129999999999999 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.204934+0000", 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024484620000000002 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207718+0000", 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002783846 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207794+0000", 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6457000000000003e-05 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.207852+0000", 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7572999999999999e-05 2024-09-13T09:26:13.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2259 3.7 3:e2f2ee12:::rbd_data.1118e3e5e25d.00000000000001d1:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.464087+0000", 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.5514068079999994, 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015117423, 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2259 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464087+0000", 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464087+0000", 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464084+0000", 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464097+0000", 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.324e-05 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464127+0000", 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9742000000000001e-05 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464140+0000", 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3405e-05 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464255+0000", 2024-09-13T09:26:13.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115028 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.464449+0000", 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019390600000000001 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.467089+0000", 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026396050000000002 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479136+0000", 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012046708 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479148+0000", 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2415000000000001e-05 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479178+0000", 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0164000000000001e-05 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479204+0000", 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5797999999999999e-05 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2258 3.17 e30/25 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head v 30'6, mlcod=29'4)", 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.470082+0000", 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.5454114830000005, 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00905918, 2024-09-13T09:26:13.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.470082+0000", 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.470082+0000", 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.470080+0000", 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471660+0000", 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015793350000000001 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471693+0000", 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3676000000000001e-05 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471709+0000", 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5832999999999999e-05 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471783+0000", 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4203999999999996e-05 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.471827+0000", 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3381999999999999e-05 2024-09-13T09:26:13.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479105+0000", 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0072785819999999996 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.479141+0000", 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5921999999999999e-05 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2265 3.2 3:43c4f885:::rbd_data.1118e3e5e25d.000000000000019f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.5258420140000002, 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017493669999999999, 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2265 2024-09-13T09:26:13.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489650+0000", 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489658+0000", 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4939999999999997e-06 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489675+0000", 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7384000000000002e-05 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489683+0000", 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.038e-06 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489735+0000", 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1824999999999999e-05 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489826+0000", 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1533999999999995e-05 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.491305+0000", 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014785320000000001 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.501618+0000", 2024-09-13T09:26:13.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010312928000000001 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.506959+0000", 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053409310000000002 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.507065+0000", 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105925 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.507145+0000", 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0455000000000001e-05 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2334 3.2 3:43c4f885:::rbd_data.1118e3e5e25d.000000000000019f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.649614+0000", 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.365879219, 2024-09-13T09:26:13.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0061120749999999998, 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2334 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649614+0000", 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649614+0000", 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649612+0000", 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649623+0000", 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0986e-05 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649640+0000", 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6441999999999999e-05 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649651+0000", 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0706999999999999e-05 2024-09-13T09:26:13.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649775+0000", 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000124735 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.649891+0000", 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115851 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:13.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.650115+0000", 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223845 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.651399+0000", 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012840779999999999 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.655671+0000", 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042715399999999999 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.655703+0000", 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2815999999999999e-05 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.655726+0000", 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2880000000000001e-05 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2815 3.1e 3:7b894b3d:::rbd_data.11189888a71f.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.1318569650000008, 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02585463, 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2815 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883635+0000", 2024-09-13T09:26:13.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883648+0000", 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3205e-05 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883682+0000", 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4421000000000001e-05 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883700+0000", 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7501000000000001e-05 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883780+0000", 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0418000000000002e-05 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883885+0000", 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105368 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.889905+0000", 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0060196479999999998 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909412+0000", 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019506672999999999 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909424+0000", 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2692e-05 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909462+0000", 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7069e-05 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909491+0000", 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9560000000000002e-05 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.108 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2922 3.7 3:e2b2cb88:::rbd_data.11189888a71f.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.119837+0000", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.8956559940000002, 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0073683689999999996, 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2922 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119837+0000", 2024-09-13T09:26:13.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119837+0000", 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119836+0000", 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119846+0000", 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7690000000000008e-06 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119868+0000", 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2212999999999999e-05 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119879+0000", 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0892e-05 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.119937+0000", 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8143999999999998e-05 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.120008+0000", 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1019000000000006e-05 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.120806+0000", 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00079801899999999996 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.125034+0000", 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042281469999999998 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.127126+0000", 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020916649999999999 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.127173+0000", 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7521000000000001e-05 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.127206+0000", 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2503999999999999e-05 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2990 3.1e 3:7b894b3d:::rbd_data.11189888a71f.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.259708+0000", 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.7557850760000004, 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012073198, 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.114 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2990 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259708+0000", 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259708+0000", 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259706+0000", 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259717+0000", 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0706999999999999e-05 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259738+0000", 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0661000000000001e-05 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259747+0000", 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9600000000000006e-06 2024-09-13T09:26:13.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259834+0000", 2024-09-13T09:26:13.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7459000000000004e-05 2024-09-13T09:26:13.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.259985+0000", 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000150901 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.262254+0000", 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022691740000000001 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271647+0000", 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093931459999999998 2024-09-13T09:26:13.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271666+0000", 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8732e-05 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271727+0000", 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0748000000000002e-05 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.271782+0000", 2024-09-13T09:26:13.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4768999999999997e-05 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3997 3.12 e35/25 3:49c74a32:::rbd_object_map.1118fb4e889d:head v 35'50, mlcod=34'48)", 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.449668+0000", 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.565825249, 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059057149999999997, 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449668+0000", 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449668+0000", 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449666+0000", 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449685+0000", 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9017000000000001e-05 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449713+0000", 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8238999999999998e-05 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.449943+0000", 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000229261 2024-09-13T09:26:13.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.450185+0000", 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000241905 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.450254+0000", 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9565000000000002e-05 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.455537+0000", 2024-09-13T09:26:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052828099999999998 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.455574+0000", 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6881000000000001e-05 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4108 3.7 3:e0c663d0:::rbd_data.1118fb4e889d.0000000000000001:head [truncate 11010048] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.5509772169999998, 2024-09-13T09:26:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0082899110000000005, 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4108 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464516+0000", 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464519+0000", 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4620000000000001e-06 2024-09-13T09:26:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464532+0000", 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2656e-05 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.464536+0000", 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465592+0000", 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001056076 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465636+0000", 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4790000000000003e-05 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465732+0000", 2024-09-13T09:26:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5775999999999999e-05 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.471690+0000", 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059574600000000004 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472734+0000", 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001044747 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472776+0000", 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1371000000000001e-05 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472806+0000", 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0332999999999999e-05 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4111 3.b 3:d2e4b237:::rbd_data.1118fb4e889d.0000000000000005:head [truncate 10747904] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.465526+0000", 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.5499676139999998, 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0072737959999999999, 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4111 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465526+0000", 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465526+0000", 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465525+0000", 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465528+0000", 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0810000000000001e-06 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465531+0000", 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.687e-06 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.465534+0000", 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8150000000000002e-06 2024-09-13T09:26:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466356+0000", 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082187700000000005 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466492+0000", 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013583099999999999 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466703+0000", 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000210912 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.471702+0000", 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004999206 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472734+0000", 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001032282 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472772+0000", 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8176000000000002e-05 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.472800+0000", 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7282999999999999e-05 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4565 3.b 3:d5801e6a:::rbd_data.11189fb5ddd6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10863232~40683] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.904991+0000", 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.1105022629999999, 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.038272699, 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4565 2024-09-13T09:26:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.904991+0000", 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.904991+0000", 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.904989+0000", 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913214+0000", 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0082248500000000006 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913250+0000", 2024-09-13T09:26:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6724999999999998e-05 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913263+0000", 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2506e-05 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913351+0000", 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8676000000000003e-05 2024-09-13T09:26:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913597+0000", 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024513900000000001 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.915014+0000", 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001417062 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.935717+0000", 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020703565 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.943108+0000", 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0073912769999999999 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.943178+0000", 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9757000000000003e-05 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.943264+0000", 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5612000000000006e-05 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4610 3.b 3:d5801e6a:::rbd_data.11189fb5ddd6.0000000000000001:head [stat,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.037430+0000", 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 2.9780634859999999, 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074137439999999999, 2024-09-13T09:26:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4610 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037430+0000", 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037430+0000", 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037426+0000", 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037446+0000", 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0112e-05 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037483+0000", 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6714000000000001e-05 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037501+0000", 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7977000000000001e-05 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037576+0000", 2024-09-13T09:26:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4993999999999994e-05 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037711+0000", 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013496999999999999 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037913+0000", 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000201566 2024-09-13T09:26:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043722+0000", 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058092329999999996 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044696+0000", 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00097460000000000005 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044779+0000", 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2348000000000005e-05 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044844+0000", 2024-09-13T09:26:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4960000000000001e-05 2024-09-13T09:26:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:13.151 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:13.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:13.153 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:13.146+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:13.253 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:13.246+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:13.270 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:13.270 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:13.270 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:13.270 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:13.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:13.338 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:26:13.339 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:26:13.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:26:13.355 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:13.350+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:13.392 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:13.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:13.392 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:13.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:13.456 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:13.449+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:13.505 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone8 intact 2024-09-13T09:26:13.511 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:461 3.1a 3:5cc0ca70:::rbd_data.111867511a81.000000000000004c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.016057+0000", 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.492739538, 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014701226, 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 461 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016057+0000", 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016057+0000", 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016056+0000", 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016062+0000", 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7130000000000002e-06 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016074+0000", 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1764e-05 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016080+0000", 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016192+0000", 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112063 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.016573+0000", 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038042799999999998 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.020585+0000", 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040121310000000004 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030630+0000", 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01004496 2024-09-13T09:26:13.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030648+0000", 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8096000000000001e-05 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030712+0000", 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4058000000000002e-05 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.030758+0000", 2024-09-13T09:26:13.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6487999999999999e-05 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1944 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 720896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100686+0000", 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.408110661, 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087232439999999998, 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1944 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100686+0000", 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100686+0000", 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100686+0000", 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100688+0000", 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0549999999999998e-06 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100693+0000", 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:26:13.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100699+0000", 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9889999999999997e-06 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109139+0000", 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084395589999999993 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109261+0000", 2024-09-13T09:26:13.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012296 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109409+0000", 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014767299999999999 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1945 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100711+0000", 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.408085831999999, 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0089238979999999996, 2024-09-13T09:26:13.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1945 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100711+0000", 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100711+0000", 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100710+0000", 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100713+0000", 2024-09-13T09:26:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2110000000000001e-06 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100718+0000", 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5639999999999996e-06 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100723+0000", 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6739999999999996e-06 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109441+0000", 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087180910000000007 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109497+0000", 2024-09-13T09:26:13.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5655999999999997e-05 2024-09-13T09:26:13.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109635+0000", 2024-09-13T09:26:13.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013803399999999999 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1946 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1638400~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100727+0000", 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.408070013, 2024-09-13T09:26:13.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092349649999999995, 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1946 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100727+0000", 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100727+0000", 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100726+0000", 2024-09-13T09:26:13.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100728+0000", 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0310000000000001e-06 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100734+0000", 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4539999999999998e-06 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100739+0000", 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2519999999999999e-06 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109694+0000", 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0089550889999999994 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109833+0000", 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013862 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109962+0000", 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012883999999999999 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1951 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100807+0000", 2024-09-13T09:26:13.533 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407990035999999, 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093793130000000002, 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1951 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100807+0000", 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100807+0000", 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100806+0000", 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100808+0000", 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100814+0000", 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2510000000000002e-06 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100818+0000", 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4120000000000003e-06 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109987+0000", 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0091692879999999994 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110059+0000", 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1972999999999997e-05 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110186+0000", 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012656000000000001 2024-09-13T09:26:13.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1952 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 753664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100822+0000", 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407974706999999, 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096213219999999999, 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1952 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100822+0000", 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100822+0000", 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100822+0000", 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100824+0000", 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0640000000000001e-06 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100829+0000", 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100833+0000", 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110221+0000", 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093875220000000006 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110279+0000", 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7923000000000001e-05 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110443+0000", 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016447 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1953 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100851+0000", 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407946000999999, 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098379939999999992, 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1953 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100851+0000", 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100851+0000", 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100850+0000", 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100857+0000", 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100863+0000", 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0220000000000001e-06 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100868+0000", 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1070000000000004e-06 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110520+0000", 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096510339999999993 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110579+0000", 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9942999999999999e-05 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110689+0000", 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109098 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1954 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100873+0000", 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407923834, 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010116484, 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1954 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100873+0000", 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100873+0000", 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100872+0000", 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100875+0000", 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100880+0000", 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2190000000000003e-06 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100885+0000", 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110723+0000", 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098385220000000006 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110853+0000", 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129484 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110989+0000", 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013664999999999999 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1959 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100949+0000", 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407847881, 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010272935, 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1959 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100949+0000", 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100949+0000", 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100948+0000", 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100951+0000", 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100957+0000", 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.409e-06 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100962+0000", 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4830000000000001e-06 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111029+0000", 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010067072999999999 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111123+0000", 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4022000000000002e-05 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111222+0000", 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8969999999999996e-05 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1960 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100966+0000", 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.40783102, 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010400461999999999, 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1960 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100966+0000", 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100966+0000", 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100965+0000", 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100967+0000", 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1160000000000002e-06 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100974+0000", 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5889999999999999e-06 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100980+0000", 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8030000000000002e-06 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111240+0000", 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010259822 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111298+0000", 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8502000000000003e-05 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111366+0000", 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7957e-05 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1961 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100984+0000", 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407812664, 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010581996999999999, 2024-09-13T09:26:13.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1961 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100984+0000", 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100984+0000", 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100983+0000", 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100991+0000", 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3610000000000003e-06 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100996+0000", 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101000+0000", 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0609999999999997e-06 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111388+0000", 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010387293000000001 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111462+0000", 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4733000000000004e-05 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111566+0000", 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103602 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.564 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1962 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.101004+0000", 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407792812, 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010711589000000001, 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1962 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.565 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101004+0000", 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101004+0000", 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101003+0000", 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101005+0000", 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.877e-06 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101010+0000", 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9880000000000004e-06 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101014+0000", 2024-09-13T09:26:13.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.019e-06 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111589+0000", 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010574717000000001 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111644+0000", 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5210000000000002e-05 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111715+0000", 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1062999999999994e-05 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1967 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.101095+0000", 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407701640999999, 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010805465, 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1967 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101095+0000", 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101095+0000", 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101095+0000", 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101101+0000", 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0599999999999996e-06 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101107+0000", 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1480000000000003e-06 2024-09-13T09:26:13.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101112+0000", 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8230000000000002e-06 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111747+0000", 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01063503 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111830+0000", 2024-09-13T09:26:13.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3651000000000003e-05 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111900+0000", 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0071e-05 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1968 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 819200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.101117+0000", 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407679273999999, 2024-09-13T09:26:13.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01100844, 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1968 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101117+0000", 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101117+0000", 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101117+0000", 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101119+0000", 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2689999999999998e-06 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101125+0000", 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9939999999999997e-06 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101134+0000", 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.969e-06 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111938+0000", 2024-09-13T09:26:13.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010803950999999999 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112037+0000", 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8682000000000001e-05 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112126+0000", 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8918000000000005e-05 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1969 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.101138+0000", 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407658257, 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011211041999999999, 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1969 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101138+0000", 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101138+0000", 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101138+0000", 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101140+0000", 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101146+0000", 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2650000000000004e-06 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101150+0000", 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1150000000000004e-06 2024-09-13T09:26:13.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112164+0000", 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011014326 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112249+0000", 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5007999999999997e-05 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112349+0000", 2024-09-13T09:26:13.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100197 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1970 3.1a 3:5ca749b4:::rbd_data.1118474c503.0000000000000002:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.101155+0000", 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.407641187999999, 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011435061, 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1970 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101155+0000", 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101155+0000", 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101155+0000", 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101157+0000", 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0379999999999998e-06 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101162+0000", 2024-09-13T09:26:13.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.101166+0000", 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112464+0000", 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011297504999999999 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112520+0000", 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6063000000000002e-05 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.112590+0000", 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0537000000000006e-05 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2831 3.3 3:c6863065:::rbd_data.11189888a71f.0000000000000008:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:04.930830+0000", 2024-09-13T09:26:13.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5779662079999994, 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099534649999999999, 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2831 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930830+0000", 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930830+0000", 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930828+0000", 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930841+0000", 2024-09-13T09:26:13.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2737e-05 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930875+0000", 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3451999999999998e-05 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930890+0000", 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5126000000000001e-05 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.930973+0000", 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3418999999999995e-05 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.931203+0000", 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000229688 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.932665+0000", 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014620670000000001 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.940604+0000", 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079391099999999992 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.940631+0000", 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6709000000000001e-05 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.940719+0000", 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8640999999999994e-05 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.940784+0000", 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4479000000000004e-05 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3063 3.9 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.1265839680000003, 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031119964999999999, 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3063 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:13.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382211+0000", 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382223+0000", 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2245e-05 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382253+0000", 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9947999999999999e-05 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382267+0000", 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4034e-05 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382338+0000", 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0931000000000002e-05 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382439+0000", 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001009 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.390414+0000", 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079749260000000002 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413261+0000", 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022847375 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413269+0000", 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413302+0000", 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3262000000000001e-05 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413333+0000", 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0077999999999999e-05 2024-09-13T09:26:13.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3101 3.9 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:05.480468+0000", 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.0283284510000001, 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019686693000000002, 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.598 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3101 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480468+0000", 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480468+0000", 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480466+0000", 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480478+0000", 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480505+0000", 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7129999999999999e-05 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480518+0000", 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3081999999999999e-05 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480614+0000", 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6047999999999999e-05 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.480761+0000", 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000146737 2024-09-13T09:26:13.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.485280+0000", 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045192619999999996 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.499968+0000", 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014688143000000001 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.500097+0000", 2024-09-13T09:26:13.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012921800000000001 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.500128+0000", 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0403999999999999e-05 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.500155+0000", 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6930000000000001e-05 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4565 3.b e35/25 3:d5801e6a:::rbd_data.11189fb5ddd6.0000000000000001:head v 35'42, mlcod=35'39)", 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.919008+0000", 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.589788457, 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026604744999999999, 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.919008+0000", 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.919008+0000", 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.919007+0000", 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.932392+0000", 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013385167 2024-09-13T09:26:13.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.932432+0000", 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0442999999999999e-05 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.932447+0000", 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4812e-05 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.932535+0000", 2024-09-13T09:26:13.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7552999999999993e-05 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.932591+0000", 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6332999999999997e-05 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.945495+0000", 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012903632999999999 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.945613+0000", 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001182 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.608 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:13.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:13.610 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:13.602+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:13.710 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:13.706+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:13.723 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:13.724 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:13.724 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:13.724 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:13.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:13.781 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T09:26:13.811 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:13.806+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:13.818 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:13.818 INFO:teuthology.orchestra.run.smithi063.stdout:881 read 0x55846a5 thru 0x558e7bd (0xa119 bytes) 2024-09-13T09:26:13.830 INFO:teuthology.orchestra.run.smithi063.stdout:882 read 0x5401000 thru 0x540efb9 (0xdfba bytes) 2024-09-13T09:26:13.831 INFO:teuthology.orchestra.run.smithi063.stdout:883 write 0x534edc2 thru 0x535579e (0x69dd bytes) 2024-09-13T09:26:13.841 INFO:teuthology.orchestra.run.smithi063.stdout:885 writesame 0x1421354 thru 0x142cc92 data_size 0x1(0xb93f bytes) 2024-09-13T09:26:13.846 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:13.846 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:13.856 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:13.871 INFO:teuthology.orchestra.run.smithi063.stdout:887 punch from 0x31fb9b9 to 0x320327b, (0x78c2 bytes) 2024-09-13T09:26:13.879 INFO:teuthology.orchestra.run.smithi063.stdout:888 trunc from 0x6727f95 to 0x95c7354 2024-09-13T09:26:13.908 INFO:teuthology.orchestra.run.smithi063.stdout:889 write 0xa2a4aba thru 0xa2afec9 (0xb410 bytes) 2024-09-13T09:26:13.912 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:13.905+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:13.915 INFO:teuthology.orchestra.run.smithi063.stdout:891 read 0x17bab16 thru 0x17c393e (0x8e29 bytes) 2024-09-13T09:26:13.926 INFO:teuthology.orchestra.run.smithi063.stdout:893 trunc from 0xa2afeca to 0x6fe0c23 2024-09-13T09:26:13.958 INFO:teuthology.orchestra.run.smithi063.stdout:894 read 0x5b37f6 thru 0x5bc79e (0x8fa9 bytes) 2024-09-13T09:26:13.970 INFO:teuthology.orchestra.run.smithi063.stdout:896 write 0x7336e2f thru 0x733e102 (0x72d4 bytes) 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:270 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:13.972 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.471076344, 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087584770000000006, 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 270 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497696+0000", 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497707+0000", 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1214e-05 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497716+0000", 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0250000000000008e-06 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497723+0000", 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504669+0000", 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069460399999999997 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504713+0000", 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3582999999999999e-05 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504887+0000", 2024-09-13T09:26:13.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174196 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505909+0000", 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001022565 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506362+0000", 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045264300000000002 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506415+0000", 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2852000000000002e-05 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506455+0000", 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0556000000000003e-05 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1851 3.1d 3:bdd77b2d:::rbd_data.1118e3e5e25d.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:13.978 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.885617311000001, 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084777419999999999, 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1851 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083154+0000", 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083170+0000", 2024-09-13T09:26:13.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6062e-05 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083198+0000", 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8191000000000002e-05 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083222+0000", 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083391+0000", 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000169276 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083553+0000", 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016206400000000001 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083765+0000", 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021140699999999999 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.085517+0000", 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001752691 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091479+0000", 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059616670000000004 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091557+0000", 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8090999999999995e-05 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091634+0000", 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6495999999999994e-05 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1976 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.86501065, 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093928829999999994, 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1976 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103762+0000", 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103767+0000", 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3550000000000004e-06 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103777+0000", 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5279999999999992e-06 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103784+0000", 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5739999999999998e-06 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106569+0000", 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002785151 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106668+0000", 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8807000000000005e-05 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107170+0000", 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050205400000000004 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109619+0000", 2024-09-13T09:26:13.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024487469999999998 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113077+0000", 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034580180000000002 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113120+0000", 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3112000000000003e-05 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113155+0000", 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5231999999999997e-05 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1977 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.864402815, 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088773199999999993, 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1977 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104369+0000", 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104390+0000", 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0999999999999999e-05 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104412+0000", 2024-09-13T09:26:13.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2248000000000001e-05 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104423+0000", 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0321e-05 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107700+0000", 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032770329999999999 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107828+0000", 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012850399999999999 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.108517+0000", 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068841499999999995 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111988+0000", 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034716550000000001 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113166+0000", 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001177893 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113215+0000", 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8758999999999999e-05 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113248+0000", 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2670999999999999e-05 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:13.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1979 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.861900742, 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085138350000000008, 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1979 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106871+0000", 2024-09-13T09:26:13.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106880+0000", 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0760000000000003e-06 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106902+0000", 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2014e-05 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106911+0000", 2024-09-13T09:26:13.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7410000000000004e-06 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110772+0000", 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038607749999999999 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110883+0000", 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011127400000000001 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:13.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111390+0000", 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050661 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115162+0000", 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037724690000000001 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115242+0000", 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0073000000000004e-05 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115303+0000", 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0631000000000003e-05 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115386+0000", 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3181000000000002e-05 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1981 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.859512747, 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083091610000000007, 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1981 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109259+0000", 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109267+0000", 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109280+0000", 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2862e-05 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109288+0000", 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7409999999999998e-06 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113375+0000", 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040869579999999999 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113466+0000", 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0756999999999999e-05 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113913+0000", 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00044651000000000001 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115567+0000", 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016543949999999999 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117501+0000", 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001933702 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117544+0000", 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3624000000000001e-05 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117570+0000", 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5347e-05 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2127 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.752595962999999, 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082727870000000002, 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2127 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216180+0000", 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.399e-06 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216188+0000", 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9699999999999999e-06 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216195+0000", 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.725e-06 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224153+0000", 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079579420000000008 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224278+0000", 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012521799999999999 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224450+0000", 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000172032 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2128 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.011 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.752572503, 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085956320000000006, 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2128 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216200+0000", 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216204+0000", 2024-09-13T09:26:14.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6399999999999999e-06 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216212+0000", 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1270000000000003e-06 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216219+0000", 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7399999999999998e-06 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224493+0000", 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082740169999999998 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224636+0000", 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014311700000000001 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224796+0000", 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016049 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2129 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.752548418, 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090836349999999996, 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2129 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216224+0000", 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216228+0000", 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.428e-06 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216235+0000", 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216242+0000", 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.973e-06 2024-09-13T09:26:14.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224846+0000", 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086038030000000001 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225053+0000", 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020684800000000001 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225308+0000", 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025549900000000001 2024-09-13T09:26:14.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2136 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.742396187000001, 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086734050000000004, 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2136 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226376+0000", 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226386+0000", 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0789e-05 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226416+0000", 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9660000000000001e-05 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226437+0000", 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0982999999999999e-05 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.229652+0000", 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032148680000000001 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.229719+0000", 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7087000000000003e-05 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.230441+0000", 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072155899999999998 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232323+0000", 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018823069999999999 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234949+0000", 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002625966 2024-09-13T09:26:14.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235010+0000", 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1101999999999999e-05 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235050+0000", 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0454999999999998e-05 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2137 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.742327060000001, 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095282279999999997, 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2137 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226450+0000", 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226463+0000", 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3338e-05 2024-09-13T09:26:14.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226467+0000", 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.481e-06 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231090+0000", 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046230450000000001 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231197+0000", 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106604 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231659+0000", 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000462189 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234788+0000", 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031288269999999998 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235890+0000", 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001101929 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235929+0000", 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8714000000000002e-05 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235974+0000", 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5611000000000001e-05 2024-09-13T09:26:14.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2138 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.742302289, 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012214620000000001, 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2138 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226474+0000", 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226485+0000", 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226501+0000", 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6671999999999999e-05 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232383+0000", 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058812739999999997 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232455+0000", 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2107999999999995e-05 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232795+0000", 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000340532 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234797+0000", 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020017889999999999 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238578+0000", 2024-09-13T09:26:14.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003780533 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238644+0000", 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5840999999999995e-05 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238686+0000", 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1984000000000003e-05 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2139 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.742262111000001, 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012363097999999999, 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2139 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226514+0000", 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.608e-06 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226519+0000", 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226523+0000", 2024-09-13T09:26:14.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9940000000000002e-06 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233419+0000", 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006896297 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233548+0000", 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012899999999999999 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234129+0000", 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058001899999999998 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237501+0000", 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033728790000000001 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238760+0000", 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001258388 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238831+0000", 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0967e-05 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238874+0000", 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3439999999999997e-05 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:14.043 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.504050345, 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:14.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.479708083, 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:14.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:14.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.475339720000001, 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:14.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:14.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:14.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1119110619999999, 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:14.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.275600581, 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:14.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:5801 3.5 e37/21 ondisk, result = 0)", 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.727147+0000", 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2416265690000001, 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012834019, 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:14.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727147+0000", 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727147+0000", 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727145+0000", 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727153+0000", 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4989999999999997e-06 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727170+0000", 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6640999999999999e-05 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727182+0000", 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1969e-05 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739763+0000", 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580996000000001 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739781+0000", 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8603000000000001e-05 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739981+0000", 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019951800000000001 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.022594553, 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:14.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:14.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.081 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:14.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:14.083 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:14.077+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:898 read 0x10e312 thru 0x113905 (0x55f4 bytes) 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:899 read 0x4830128 thru 0x483e4cd (0xe3a6 bytes) 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:901 write 0xa22281d thru 0xa22675e (0x3f42 bytes) 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:902 writesame 0x86b360c thru 0x86b8e9d data_size 0x3(0x5892 bytes) 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:903 punch from 0x3b5063e to 0x3b5a7c6, (0xa188 bytes) 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:904 trunc from 0xa22675f to 0xddfe43a 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.084 INFO:teuthology.orchestra.run.smithi063.stdout:907 trunc from 0xddfe43a to 0x2675f70 2024-09-13T09:26:14.085 INFO:teuthology.orchestra.run.smithi063.stdout:908 read 0xc9cb6 thru 0xd17d8 (0x7b23 bytes) 2024-09-13T09:26:14.110 INFO:teuthology.orchestra.run.smithi063.stdout:909 trunc from 0x2675f70 to 0xe1d226a 2024-09-13T09:26:14.123 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.123 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.124 INFO:teuthology.orchestra.run.smithi063.stdout:912 read 0xde54727 thru 0xde63492 (0xed6c bytes) 2024-09-13T09:26:14.124 INFO:teuthology.orchestra.run.smithi063.stdout:913 punch from 0x49f262 to 0x49faae, (0x84c bytes) 2024-09-13T09:26:14.124 INFO:teuthology.orchestra.run.smithi063.stdout:917 punch from 0x8cf67ef to 0x8d05c6b, (0xf47c bytes) 2024-09-13T09:26:14.124 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.124 INFO:teuthology.orchestra.run.smithi063.stdout:922 write 0x2730795 thru 0x273c881 (0xc0ed bytes) 2024-09-13T09:26:14.127 INFO:teuthology.orchestra.run.smithi063.stdout:924 write 0x90f71d4 thru 0x9101af5 (0xa922 bytes) 2024-09-13T09:26:14.135 INFO:teuthology.orchestra.run.smithi063.stdout:925 punch from 0x829344e to 0x8298001, (0x4bb3 bytes) 2024-09-13T09:26:14.135 INFO:teuthology.orchestra.run.smithi063.stdout:927 writesame 0x240da51 thru 0x241ca88 data_size 0x4(0xf038 bytes) 2024-09-13T09:26:14.151 INFO:teuthology.orchestra.run.smithi063.stdout:928 punch from 0x172559c to 0x1727661, (0x20c5 bytes) 2024-09-13T09:26:14.151 INFO:teuthology.orchestra.run.smithi063.stdout:931 write 0x17bb8a4 thru 0x17ca6a6 (0xee03 bytes) 2024-09-13T09:26:14.154 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.154 INFO:teuthology.orchestra.run.smithi063.stdout:933 read 0xf49de8 thru 0xf59d2d (0xff46 bytes) 2024-09-13T09:26:14.176 INFO:teuthology.orchestra.run.smithi063.stdout:934 trunc from 0xe1d226a to 0x480170f 2024-09-13T09:26:14.183 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:14.177+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:14.198 INFO:teuthology.orchestra.run.smithi063.stdout:935 read 0x109b4be thru 0x10a5a92 (0xa5d5 bytes) 2024-09-13T09:26:14.199 INFO:teuthology.orchestra.run.smithi063.stdout:937 read 0x2baea26 thru 0x2bb59ee (0x6fc9 bytes) 2024-09-13T09:26:14.200 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.220 INFO:teuthology.orchestra.run.smithi063.stdout:939 trunc from 0x480170f to 0xd0e2a62 2024-09-13T09:26:14.233 INFO:teuthology.orchestra.run.smithi063.stdout:940 read 0x3b93ebc thru 0x3b96bf2 (0x2d37 bytes) 2024-09-13T09:26:14.233 INFO:teuthology.orchestra.run.smithi063.stdout:941 write 0xb583684 thru 0xb591ff3 (0xe970 bytes) 2024-09-13T09:26:14.242 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:14.242 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:14.242 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:14.242 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:14.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:14.253 INFO:teuthology.orchestra.run.smithi063.stdout:942 writesame 0xda312e8 thru 0xda3e187 data_size 0x3a(0xcea0 bytes) 2024-09-13T09:26:14.253 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.253 INFO:teuthology.orchestra.run.smithi063.stdout:945 write 0x9da29d0 thru 0x9dab46d (0x8a9e bytes) 2024-09-13T09:26:14.259 INFO:teuthology.orchestra.run.smithi063.stdout:946 read 0x8c0c377 thru 0x8c14d29 (0x89b3 bytes) 2024-09-13T09:26:14.259 INFO:teuthology.orchestra.run.smithi063.stdout:948 read 0xb68450c thru 0xb68bd2d (0x7822 bytes) 2024-09-13T09:26:14.259 INFO:teuthology.orchestra.run.smithi063.stdout:949 writesame 0x38057c0 thru 0x38119db data_size 0x65(0xc21c bytes) 2024-09-13T09:26:14.266 INFO:teuthology.orchestra.run.smithi063.stdout:950 writesame 0x5151743 thru 0x5153092 data_size 0x3c(0x1950 bytes) 2024-09-13T09:26:14.273 INFO:teuthology.orchestra.run.smithi063.stdout:951 write 0x96c598a thru 0x96d2ece (0xd545 bytes) 2024-09-13T09:26:14.281 INFO:teuthology.orchestra.run.smithi063.stdout:952 read 0xd906d27 thru 0xd910691 (0x996b bytes) 2024-09-13T09:26:14.281 INFO:teuthology.orchestra.run.smithi063.stdout:953 read 0xa1bc779 thru 0xa1c74a3 (0xad2b bytes) 2024-09-13T09:26:14.281 INFO:teuthology.orchestra.run.smithi063.stdout:954 punch from 0xcae6ddd to 0xcaf4321, (0xd544 bytes) 2024-09-13T09:26:14.281 INFO:teuthology.orchestra.run.smithi063.stdout:956 write 0x144b76 thru 0x14dfef (0x947a bytes) 2024-09-13T09:26:14.284 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.284 INFO:teuthology.orchestra.run.smithi063.stdout:958 write 0x68480ea thru 0x685523d (0xd154 bytes) 2024-09-13T09:26:14.284 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:14.278+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:14.290 INFO:teuthology.orchestra.run.smithi063.stdout:959 write 0x88c6269 thru 0x88c9c22 (0x39ba bytes) 2024-09-13T09:26:14.297 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.298 INFO:teuthology.orchestra.run.smithi063.stdout:961 trunc from 0xda3e188 to 0xd2ceb8c 2024-09-13T09:26:14.311 INFO:teuthology.orchestra.run.smithi063.stdout:964 writesame 0xc9bd929 thru 0xc9c9613 data_size 0x8d(0xbceb bytes) 2024-09-13T09:26:14.329 INFO:teuthology.orchestra.run.smithi063.stdout:965 trunc from 0xd2ceb8c to 0x97d118e 2024-09-13T09:26:14.353 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:26:14.364 INFO:teuthology.orchestra.run.smithi063.stdout:966 trunc from 0x97d118e to 0xa0a468c 2024-09-13T09:26:14.367 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:26:14.368 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:14.368 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:14.368 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:14.368 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:14.368 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:14.377 INFO:teuthology.orchestra.run.smithi063.stdout:967 read 0x9ecdfff thru 0x9edc43a (0xe43c bytes) 2024-09-13T09:26:14.377 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:14.377 INFO:teuthology.orchestra.run.smithi063.stdout:969 write 0x94b657e thru 0x94c305c (0xcadf bytes) 2024-09-13T09:26:14.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:14.379 INFO:teuthology.orchestra.run.smithi063.stdout:970 clone 10 order 20 su 4096 sc 12 2024-09-13T09:26:14.385 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:14.378+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:14.482 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:14.482 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:199 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.164617293999999, 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060758599999999998, 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 199 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313898+0000", 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313896+0000", 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313906+0000", 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1870000000000003e-06 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313920+0000", 2024-09-13T09:26:14.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3871e-05 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.313940+0000", 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0582999999999999e-05 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.314070+0000", 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013007399999999999 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.314087+0000", 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6413e-05 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317280+0000", 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003193244 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317574+0000", 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029457700000000002 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.317844+0000", 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000269831 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319224+0000", 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001379665 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319823+0000", 2024-09-13T09:26:14.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059955700000000002 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319911+0000", 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.319974+0000", 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2651999999999996e-05 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:260 3.a 3:55f6faff:::rbd_data.111867511a81.00000000000000eb:head [set-alloc-hint object_size 1048576 write_size 1048576,write 600819~13703] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.021003840000001, 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002930972, 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 260 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457512+0000", 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457509+0000", 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457561+0000", 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1952e-05 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457595+0000", 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4269e-05 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457620+0000", 2024-09-13T09:26:14.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.459e-05 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.457750+0000", 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013036699999999999 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458002+0000", 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025142099999999999 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.458263+0000", 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026117700000000002 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.459691+0000", 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014285159999999999 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460298+0000", 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00060671200000000003 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460383+0000", 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4834999999999997e-05 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.460443+0000", 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9616e-05 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:14.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.919385195, 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:14.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:297 3.10 e27/25 ondisk, result = 0)", 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.877528271999999, 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006064926, 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600987+0000", 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600986+0000", 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.600992+0000", 2024-09-13T09:26:14.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7699999999999998e-06 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.601009+0000", 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7197000000000002e-05 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.601020+0000", 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0648999999999999e-05 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606853+0000", 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058327409999999998 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606867+0000", 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.43e-05 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607052+0000", 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018516600000000001 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:324 3.a 3:51773985:::rbd_data.111867511a81.00000000000000ca:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.801454530999999, 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003713062, 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 324 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677061+0000", 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677059+0000", 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677071+0000", 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2133000000000001e-05 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677095+0000", 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4015e-05 2024-09-13T09:26:14.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677106+0000", 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0542e-05 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677216+0000", 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110306 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677397+0000", 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181056 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.677562+0000", 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016472500000000001 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.678899+0000", 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013370470000000001 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680639+0000", 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001739864 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680712+0000", 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3681000000000001e-05 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.680774+0000", 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865000000000005e-05 2024-09-13T09:26:14.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:347 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.766074191, 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045695639999999999, 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 347 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712441+0000", 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712439+0000", 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712454+0000", 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5228000000000001e-05 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712490+0000", 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5577000000000002e-05 2024-09-13T09:26:14.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712517+0000", 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7582000000000001e-05 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712595+0000", 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8011999999999998e-05 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712678+0000", 2024-09-13T09:26:14.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2559999999999996e-05 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.712778+0000", 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9698000000000007e-05 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.714327+0000", 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001549784 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716842+0000", 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025141489999999998 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.716940+0000", 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8374000000000004e-05 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.717011+0000", 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1023000000000001e-05 2024-09-13T09:26:14.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:518 3.1e e27/21 3:7c1f840a:::rbd_data.111867511a81.00000000000000d0:head v 27'7, mlcod=27'5)", 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.341867410000001, 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031587289999999999, 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136648+0000", 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136646+0000", 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136823+0000", 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017679600000000001 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136851+0000", 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8195e-05 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136870+0000", 2024-09-13T09:26:14.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.931e-05 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.136992+0000", 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121146 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.137050+0000", 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8142e-05 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.139718+0000", 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026680380000000002 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:00.139807+0000", 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9165000000000004e-05 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:1862 3.1e e30/21 3:7913b972:::rbd_data.1118e3e5e25d.00000000000000c7:head v 30'13, mlcod=30'11)", 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.087793+0000", 2024-09-13T09:26:14.522 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.390722675999999, 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003437973, 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087793+0000", 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087793+0000", 2024-09-13T09:26:14.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087791+0000", 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087805+0000", 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3902e-05 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087834+0000", 2024-09-13T09:26:14.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8345000000000001e-05 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087850+0000", 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6512e-05 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087937+0000", 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6398999999999997e-05 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.087993+0000", 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6820999999999997e-05 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091183+0000", 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031895890000000001 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091231+0000", 2024-09-13T09:26:14.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7895000000000001e-05 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:1876 3.1e e30/21 3:7d1bc93e:::rbd_data.1118e3e5e25d.000000000000000e:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.100184+0000", 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.378331915, 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003290608, 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100184+0000", 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100184+0000", 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100183+0000", 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100194+0000", 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1018e-05 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100210+0000", 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6018000000000001e-05 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100218+0000", 2024-09-13T09:26:14.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1019999999999997e-06 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100279+0000", 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1793999999999998e-05 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.100316+0000", 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6248000000000003e-05 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103430+0000", 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003114688 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103474+0000", 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3841000000000003e-05 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:2143 3.1e e30/21 3:7a2e114f:::rbd_data.1118e3e5e25d.0000000000000009:head v 30'18, mlcod=30'16)", 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.234310+0000", 2024-09-13T09:26:14.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.244205236999999, 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030434139999999999, 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234310+0000", 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234310+0000", 2024-09-13T09:26:14.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234309+0000", 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234318+0000", 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9420000000000001e-06 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234335+0000", 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6657e-05 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234350+0000", 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5146e-05 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234419+0000", 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9595e-05 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234441+0000", 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1871000000000001e-05 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237317+0000", 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028752579999999999 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237354+0000", 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7082000000000002e-05 2024-09-13T09:26:14.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2218 3.10 3:0a24612b:::rbd_data.1118e3e5e25d.00000000000002a2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.371303+0000", 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.107212878, 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039573680000000002, 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2218 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371303+0000", 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371303+0000", 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371300+0000", 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371316+0000", 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371355+0000", 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9035999999999997e-05 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371375+0000", 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9811000000000001e-05 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371448+0000", 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2527999999999994e-05 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371527+0000", 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9525999999999997e-05 2024-09-13T09:26:14.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.371625+0000", 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7709999999999998e-05 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.373856+0000", 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022314370000000002 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.375176+0000", 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013200659999999999 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.375221+0000", 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4209999999999999e-05 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.375260+0000", 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9338000000000002e-05 2024-09-13T09:26:14.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2332 3.10 3:08708bba:::rbd_data.1118e3e5e25d.000000000000014f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.652283+0000", 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.826232812000001, 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061894089999999999, 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2332 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652283+0000", 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652283+0000", 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652280+0000", 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652296+0000", 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5418999999999999e-05 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652326+0000", 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0602999999999998e-05 2024-09-13T09:26:14.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652342+0000", 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5486e-05 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652552+0000", 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000209846 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.652822+0000", 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027073700000000001 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.653090+0000", 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000267724 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.654470+0000", 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001379786 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.658307+0000", 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003836711 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.658404+0000", 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6921000000000003e-05 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.658472+0000", 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8582000000000002e-05 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2507 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:04.392672+0000", 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.085843169, 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0063454289999999997, 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2507 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392672+0000", 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392672+0000", 2024-09-13T09:26:14.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392670+0000", 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392686+0000", 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6663e-05 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392722+0000", 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5902000000000003e-05 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392738+0000", 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5783000000000001e-05 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392844+0000", 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010608299999999999 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.392852+0000", 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7700000000000001e-06 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.396421+0000", 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003569352 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.396602+0000", 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000180299 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.396966+0000", 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036429600000000001 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.398478+0000", 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001512003 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.398873+0000", 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039520400000000002 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.398958+0000", 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4911000000000001e-05 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.399018+0000", 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9825999999999999e-05 2024-09-13T09:26:14.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2523 3.a 3:535385f0:::rbd_children:head [call rbd.remove_child in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:04.420048+0000", 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.058467919, 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028996289999999999, 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2523 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420048+0000", 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420048+0000", 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420045+0000", 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420061+0000", 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6336000000000001e-05 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420097+0000", 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6115000000000002e-05 2024-09-13T09:26:14.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420113+0000", 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5444e-05 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420218+0000", 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105264 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420365+0000", 2024-09-13T09:26:14.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014688599999999999 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.420671+0000", 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030591700000000001 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.421899+0000", 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012280430000000001 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.422709+0000", 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00080991500000000003 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.422849+0000", 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013985399999999999 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:04.422947+0000", 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8474000000000006e-05 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3686 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:08.408394+0000", 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.0701212240000002, 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0062503699999999999, 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3686 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408394+0000", 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408394+0000", 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408392+0000", 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408420+0000", 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8166000000000001e-05 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408456+0000", 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5695999999999998e-05 2024-09-13T09:26:14.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408479+0000", 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3906000000000001e-05 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408569+0000", 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9623000000000006e-05 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.408575+0000", 2024-09-13T09:26:14.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1349999999999997e-06 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.412408+0000", 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003832851 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.412489+0000", 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1012000000000003e-05 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.412684+0000", 2024-09-13T09:26:14.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019534199999999999 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.413920+0000", 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012356369999999999 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.414571+0000", 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00065120399999999995 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.414612+0000", 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0901e-05 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.414645+0000", 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2564000000000002e-05 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:3703 3.1e e34/21 3:7bc75c0a:::rbd_data.111851704b2b.0000000000000009:head v 34'33, mlcod=32'30)", 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:08.438242+0000", 2024-09-13T09:26:14.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.0402735099999996, 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029243889999999999, 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.438242+0000", 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.438242+0000", 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.438241+0000", 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.438253+0000", 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2041999999999999e-05 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.438285+0000", 2024-09-13T09:26:14.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2036999999999998e-05 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.438301+0000", 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5262000000000001e-05 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.439006+0000", 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00070549700000000003 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.439059+0000", 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2745999999999999e-05 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.441107+0000", 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020485709999999999 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:08.441166+0000", 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9030999999999998e-05 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3952 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.428434+0000", 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.050081595, 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00522946, 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3952 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.572 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428434+0000", 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428434+0000", 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428431+0000", 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428448+0000", 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7329999999999998e-05 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428485+0000", 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6455999999999998e-05 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428502+0000", 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7249e-05 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428630+0000", 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000128011 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.428644+0000", 2024-09-13T09:26:14.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.432150+0000", 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035061390000000001 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.432214+0000", 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3402e-05 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.432361+0000", 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014706699999999999 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.433356+0000", 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099507999999999997 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.433600+0000", 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024404299999999999 2024-09-13T09:26:14.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.433636+0000", 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5771e-05 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.433663+0000", 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7838999999999999e-05 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6379 3.10 3:0e7ac917:::rbd_data.1118e98b86fc.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,truncate 2883584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:13.943237+0000", 2024-09-13T09:26:14.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.53527877999999995, 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046310070000000004, 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6379 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943237+0000", 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943237+0000", 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943235+0000", 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943245+0000", 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.149e-06 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943262+0000", 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7764000000000001e-05 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943269+0000", 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6499999999999999e-06 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943356+0000", 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6643000000000004e-05 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943605+0000", 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024917500000000003 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.943839+0000", 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234274 2024-09-13T09:26:14.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.945122+0000", 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012829650000000001 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.947782+0000", 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026596060000000001 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.947834+0000", 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1989e-05 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.947868+0000", 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4186000000000001e-05 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6433 3.10 3:0e7ac917:::rbd_data.1118e98b86fc.0000000000000019:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:14.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:14.055316+0000", 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42319931700000002, 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042603390000000001, 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6433 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055316+0000", 2024-09-13T09:26:14.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055316+0000", 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055313+0000", 2024-09-13T09:26:14.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055331+0000", 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.719e-05 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055376+0000", 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5615999999999998e-05 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055401+0000", 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4406e-05 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055572+0000", 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001717 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055869+0000", 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029634600000000001 2024-09-13T09:26:14.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.056136+0000", 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000267197 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.058829+0000", 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00269323 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.059424+0000", 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059474699999999996 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.059500+0000", 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5975000000000004e-05 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.059577+0000", 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6761999999999995e-05 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:6434 3.1e e38/21 3:7c035b83:::rbd_data.1118e98b86fc.000000000000001a:head v 38'35, mlcod=34'33)", 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:14.055905+0000", 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.42261086599999997, 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033745210000000001, 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055905+0000", 2024-09-13T09:26:14.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055905+0000", 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055903+0000", 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055921+0000", 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8377e-05 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055944+0000", 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3339000000000001e-05 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.055978+0000", 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3796e-05 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.056248+0000", 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026984099999999999 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.056314+0000", 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6018000000000004e-05 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.059185+0000", 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028711259999999999 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.059279+0000", 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3948000000000002e-05 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.596 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:14.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:14.598 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:14.597+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:14.698 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:14.694+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:14.713 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:14.713 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:14.713 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:14.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:14.738 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:14.800 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:14.797+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:14.827 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:14.828 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:14.828 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:14.828 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:14.828 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:14.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:14.890 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:14.901 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:14.894+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:14.912 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:14.929 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:14.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.984818099000002, 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:14.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:14.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.988428643999999, 2024-09-13T09:26:14.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:14.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:14.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.579855011999999, 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014920113, 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.963 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365977+0000", 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365987+0000", 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366004+0000", 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.662e-05 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366022+0000", 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7883000000000001e-05 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376091+0000", 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010069171999999999 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376246+0000", 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000155377 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376548+0000", 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030137300000000002 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380798+0000", 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004250406 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380813+0000", 2024-09-13T09:26:14.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4357e-05 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380862+0000", 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9137000000000001e-05 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380898+0000", 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6158999999999997e-05 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.579796433999999, 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016811104, 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366035+0000", 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366045+0000", 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0327e-05 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366062+0000", 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6483000000000001e-05 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366074+0000", 2024-09-13T09:26:14.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377874+0000", 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011799983999999999 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378006+0000", 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013130100000000001 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378143+0000", 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137157 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380914+0000", 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002771281 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382760+0000", 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001846253 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382810+0000", 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9483e-05 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382847+0000", 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7749000000000001e-05 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.974 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.579716732000001, 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018719919000000002, 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366115+0000", 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366313+0000", 2024-09-13T09:26:14.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000198156 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366331+0000", 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7739000000000001e-05 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366345+0000", 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4226000000000001e-05 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378684+0000", 2024-09-13T09:26:14.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012339325 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378827+0000", 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014284700000000001 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.379116+0000", 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028930399999999999 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382675+0000", 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035590050000000001 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384568+0000", 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001892707 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384706+0000", 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013804099999999999 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384836+0000", 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012977900000000001 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.579470573999998, 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01866932, 2024-09-13T09:26:14.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366361+0000", 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366372+0000", 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0857e-05 2024-09-13T09:26:14.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366388+0000", 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366401+0000", 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.326e-05 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380005+0000", 2024-09-13T09:26:14.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013604036 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380100+0000", 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4468000000000004e-05 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380211+0000", 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011084199999999999 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383662+0000", 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034510460000000001 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384905+0000", 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001243009 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384996+0000", 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:26:14.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385032+0000", 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5528000000000003e-05 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.579417724999999, 2024-09-13T09:26:14.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020964449999999999, 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366414+0000", 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366423+0000", 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:26:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366443+0000", 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366455+0000", 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380966+0000", 2024-09-13T09:26:14.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014510662000000001 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381062+0000", 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6526000000000004e-05 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381259+0000", 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019658499999999999 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383675+0000", 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024160480000000001 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387295+0000", 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036198020000000001 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387342+0000", 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7299000000000002e-05 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387380+0000", 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7379000000000002e-05 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.578993698000001, 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020678247, 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366838+0000", 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367071+0000", 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023322800000000001 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367094+0000", 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3079999999999999e-05 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367111+0000", 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7008999999999999e-05 2024-09-13T09:26:14.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381788+0000", 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014676850999999999 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381894+0000", 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010610799999999999 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382045+0000", 2024-09-13T09:26:14.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015073699999999999 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383689+0000", 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016436630000000001 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387445+0000", 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037564 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:14.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387488+0000", 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3402000000000002e-05 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387517+0000", 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8975999999999998e-05 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:14.999 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.578704010999999, 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020557408999999999, 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:15.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367128+0000", 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367138+0000", 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0672999999999999e-05 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367157+0000", 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8779000000000001e-05 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367174+0000", 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6464e-05 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382926+0000", 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015752759000000002 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383077+0000", 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150965 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383228+0000", 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015101399999999999 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384458+0000", 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012294370000000001 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387599+0000", 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031411960000000002 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387647+0000", 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7648999999999997e-05 2024-09-13T09:26:15.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387686+0000", 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9653000000000001e-05 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.005 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.578644906000001, 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022933500999999998, 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367187+0000", 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367208+0000", 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1793e-05 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367225+0000", 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668999999999999e-05 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367239+0000", 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4005999999999999e-05 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383748+0000", 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016508719000000002 2024-09-13T09:26:15.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383933+0000", 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018481500000000001 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384122+0000", 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189626 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385049+0000", 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092678599999999997 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390003+0000", 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049541819999999997 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390075+0000", 2024-09-13T09:26:15.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2094999999999993e-05 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390121+0000", 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6082999999999999e-05 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.012 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988589525999998, 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015204896000000001, 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 135 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957242+0000", 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957250+0000", 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957268+0000", 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7459999999999999e-05 2024-09-13T09:26:15.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957277+0000", 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3559999999999999e-06 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965456+0000", 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792359999999995 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965567+0000", 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110877 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965741+0000", 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017349600000000001 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967304+0000", 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001563444 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972380+0000", 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050761019999999999 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972425+0000", 2024-09-13T09:26:15.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4366000000000002e-05 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972448+0000", 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3455000000000002e-05 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:15.019 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988545456000001, 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015208101, 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 136 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957297+0000", 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9909999999999997e-06 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957339+0000", 2024-09-13T09:26:15.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2267e-05 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957371+0000", 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2518000000000003e-05 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966255+0000", 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088832679999999997 2024-09-13T09:26:15.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966353+0000", 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8658000000000003e-05 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966486+0000", 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013309500000000001 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969257+0000", 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027707209999999999 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972453+0000", 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031962789999999998 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972478+0000", 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4655e-05 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972495+0000", 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7407000000000001e-05 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988430595000001, 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015156871000000001, 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-13T09:26:15.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957401+0000", 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957509+0000", 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010808 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957536+0000", 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6237e-05 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957542+0000", 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2020000000000001e-06 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967352+0000", 2024-09-13T09:26:15.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098099580000000006 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967447+0000", 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5375000000000001e-05 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967584+0000", 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136463 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971541+0000", 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039574930000000003 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972499+0000", 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095777799999999995 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972527+0000", 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7900999999999999e-05 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972559+0000", 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2191999999999999e-05 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988280757999998, 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015104328, 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957551+0000", 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957556+0000", 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.318e-06 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957565+0000", 2024-09-13T09:26:15.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.115e-06 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957572+0000", 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968300+0000", 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010727617 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968387+0000", 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7021000000000006e-05 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968482+0000", 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5081999999999995e-05 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971595+0000", 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031126639999999998 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972588+0000", 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099296600000000007 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972635+0000", 2024-09-13T09:26:15.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7136999999999999e-05 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972656+0000", 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1480000000000001e-05 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.040 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988251956999999, 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015429492, 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.041 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957580+0000", 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957586+0000", 2024-09-13T09:26:15.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957620+0000", 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3215e-05 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957651+0000", 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1027e-05 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969570+0000", 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011919511000000001 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969674+0000", 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104248 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969784+0000", 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010937700000000001 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972662+0000", 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028781589999999999 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972925+0000", 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026289899999999998 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972976+0000", 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0702999999999999e-05 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973010+0000", 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4743000000000003e-05 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:15.048 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988147352999999, 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020098000000000001, 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957793+0000", 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108837 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957808+0000", 2024-09-13T09:26:15.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957814+0000", 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970251+0000", 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012436523999999999 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970350+0000", 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9600999999999997e-05 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970506+0000", 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015539500000000001 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972667+0000", 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002161619 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977681+0000", 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050135930000000002 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977722+0000", 2024-09-13T09:26:15.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1634999999999997e-05 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977783+0000", 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0968000000000003e-05 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.988011226000001, 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022700502000000001, 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957821+0000", 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957825+0000", 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957843+0000", 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7492000000000001e-05 2024-09-13T09:26:15.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957851+0000", 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971666+0000", 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013814707000000001 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971841+0000", 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174659 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971976+0000", 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013485600000000001 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973161+0000", 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011852130000000001 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980427+0000", 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072666440000000001 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980487+0000", 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9373000000000002e-05 2024-09-13T09:26:15.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980522+0000", 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5244999999999999e-05 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:4479 3.d 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:15.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.110072679, 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051456631000000003, 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4479 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835751+0000", 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999909 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854271+0000", 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01851972 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854305+0000", 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4258000000000003e-05 2024-09-13T09:26:15.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854317+0000", 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2013e-05 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854487+0000", 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017018 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854768+0000", 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028067700000000001 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.855666+0000", 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000898404 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.881583+0000", 2024-09-13T09:26:15.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025916947999999999 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887097+0000", 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055134479999999998 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887161+0000", 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4077000000000003e-05 2024-09-13T09:26:15.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887217+0000", 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6128000000000002e-05 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6281 3.13 3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:15.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.098805874, 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024029966, 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6281 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847025+0000", 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:15.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847238+0000", 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021285799999999999 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847259+0000", 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0908e-05 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847267+0000", 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.67e-06 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847329+0000", 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2016999999999999e-05 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847464+0000", 2024-09-13T09:26:15.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134497 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.853158+0000", 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056936219999999997 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870613+0000", 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017455404000000001 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870743+0000", 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012992099999999999 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870911+0000", 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168276 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.871057+0000", 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014575999999999999 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6591 3.13 3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.81030326600000002, 2024-09-13T09:26:15.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015116329, 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6591 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135528+0000", 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:15.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135537+0000", 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0669999999999998e-06 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135554+0000", 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135563+0000", 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5380000000000008e-06 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135634+0000", 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1106999999999995e-05 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:15.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135759+0000", 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012423999999999999 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.139009+0000", 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032509520000000001 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150356+0000", 2024-09-13T09:26:15.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011346962 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150461+0000", 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104998 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150550+0000", 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8622999999999995e-05 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:15.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150646+0000", 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5822000000000005e-05 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:15.083 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:15.085 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:15.085 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:15.085 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:15.086 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:15.086 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:15.086 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:15.081+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:15.087 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:15.087 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:15.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:15.185 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:15.185+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:15.205 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.205 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:15.286 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:15.285+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:15.320 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.320 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.331 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:15.353 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:26:15.368 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:26:15.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:26:15.387 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:15.385+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:15.446 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.446 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.461 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:15.488 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:15.485+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:15.550 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone9 from 0xa0a468c (overlap 0x2675f70) to 0x0 2024-09-13T09:26:15.571 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.571 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.582 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:15.589 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:15.589+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:15.685 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.685 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.690 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:15.689+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:15.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:15.791 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:15.787+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:15.792 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T09:26:15.808 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.808 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.818 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:15.893 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:15.887+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:15.929 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:15.930 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:15.940 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:15.994 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:15.991+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:15.995 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:15.995 INFO:teuthology.orchestra.run.smithi063.stdout:972 punch from 0x7c65719 to 0x7c6ec1c, (0x9503 bytes) 2024-09-13T09:26:16.003 INFO:teuthology.orchestra.run.smithi063.stdout:974 punch from 0x7f7acf7 to 0x7f83d6e, (0x9077 bytes) 2024-09-13T09:26:16.003 INFO:teuthology.orchestra.run.smithi063.stdout:975 read 0x6586604 thru 0x659059a (0x9f97 bytes) 2024-09-13T09:26:16.009 INFO:teuthology.orchestra.run.smithi063.stdout:976 read 0x7963afc thru 0x7967cfa (0x41ff bytes) 2024-09-13T09:26:16.095 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:16.091+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:16.113 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.113 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:16.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:16.147 INFO:teuthology.orchestra.run.smithi063.stdout:977 write 0xb622604 thru 0xb622869 (0x266 bytes) 2024-09-13T09:26:16.152 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:16.152 INFO:teuthology.orchestra.run.smithi063.stdout:980 write 0x6c3e4ec thru 0x6c4d9d3 (0xf4e8 bytes) 2024-09-13T09:26:16.196 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:16.191+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:16.260 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.260 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:16.271 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:16.282 INFO:teuthology.orchestra.run.smithi063.stdout:982 write 0xde1e6b8 thru 0xde25e1c (0x7765 bytes) 2024-09-13T09:26:16.296 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:16.293+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:16.314 INFO:teuthology.orchestra.run.smithi063.stdout:983 trunc from 0xde25e1d to 0x83bda02 2024-09-13T09:26:16.377 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:26:16.378 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:26:16.395 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:26:16.397 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:16.397+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:16.403 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.403 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:16.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:16.427 INFO:teuthology.orchestra.run.smithi063.stdout:984 trunc from 0x83bda02 to 0x32f5aa7 2024-09-13T09:26:16.498 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:16.497+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:16.541 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.541 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:16.551 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:16.569 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:16.570 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:16.589 INFO:teuthology.orchestra.run.smithi063.stdout:987 write 0xa1a0f79 thru 0xa1a3fa4 (0x302c bytes) 2024-09-13T09:26:16.599 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:16.597+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:16.600 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:990 read 0x48f7499 thru 0x4901151 (0x9cb9 bytes) 2024-09-13T09:26:16.601 INFO:teuthology.orchestra.run.smithi063.stdout:991 writesame 0x6c76a54 thru 0x6c7da9d data_size 0x9(0x704a bytes) 2024-09-13T09:26:16.620 INFO:teuthology.orchestra.run.smithi063.stdout:992 read 0x2946681 thru 0x2952bea (0xc56a bytes) 2024-09-13T09:26:16.666 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.666 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:16.678 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:16.709 INFO:teuthology.orchestra.run.smithi063.stdout:993 write 0x1f6dd68 thru 0x1f76b36 (0x8dcf bytes) 2024-09-13T09:26:16.725 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:16.725+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:16.803 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:16.797+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:16.864 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.864 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:16.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:16.902 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:16.901+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:16.925 INFO:teuthology.orchestra.run.smithi063.stdout:995 write 0xe0528be thru 0xe055667 (0x2daa bytes) 2024-09-13T09:26:16.937 INFO:teuthology.orchestra.run.smithi063.stdout:998 punch from 0x9ed4d9b to 0x9ee13ec, (0xc651 bytes) 2024-09-13T09:26:16.937 INFO:teuthology.orchestra.run.smithi063.stdout:999 punch from 0x11f1234 to 0x11f8757, (0x7523 bytes) 2024-09-13T09:26:16.937 INFO:teuthology.orchestra.run.smithi063.stdout:1000 write 0xcfce91a thru 0xcfdcf51 (0xe638 bytes) 2024-09-13T09:26:16.957 INFO:teuthology.orchestra.run.smithi063.stdout:1002 read 0x20c91d0 thru 0x20d65e5 (0xd416 bytes) 2024-09-13T09:26:16.984 INFO:teuthology.orchestra.run.smithi063.stdout:1003 trunc from 0xe055668 to 0x5809ff1 2024-09-13T09:26:16.991 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:16.992 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:17.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:17.004 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:17.001+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:17.014 INFO:teuthology.orchestra.run.smithi063.stdout:1004 read 0x37b5f47 thru 0x37be4b6 (0x8570 bytes) 2024-09-13T09:26:17.015 INFO:teuthology.orchestra.run.smithi063.stdout:1005 punch from 0x142a210 to 0x142dba5, (0x3995 bytes) 2024-09-13T09:26:17.015 INFO:teuthology.orchestra.run.smithi063.stdout:1006 flatten 2024-09-13T09:26:17.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:17.103 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:17.099+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:17.204 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:17.201+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:17.242 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:17.242 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:17.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:17.258 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:17.284 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:17.305 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:17.301+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:17.307 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:17.321 INFO:teuthology.orchestra.run.smithi063.stdout:1007 read 0x46cd80a thru 0x46d8929 (0xb120 bytes) 2024-09-13T09:26:17.328 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:17.329 INFO:teuthology.orchestra.run.smithi063.stdout:1008 trunc from 0x5809ff1 to 0x8e76a07 2024-09-13T09:26:17.337 INFO:teuthology.orchestra.run.smithi063.stdout:1009 punch from 0x1c5a293 to 0x1c6154c, (0x72b9 bytes) 2024-09-13T09:26:17.346 INFO:teuthology.orchestra.run.smithi063.stdout:1010 trunc from 0x8e76a07 to 0xd49fd35 2024-09-13T09:26:17.349 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:17.361 INFO:teuthology.orchestra.run.smithi063.stdout:1011 trunc from 0xd49fd35 to 0xa51ae29 2024-09-13T09:26:17.372 INFO:teuthology.orchestra.run.smithi063.stdout:1013 read 0xb3caa thru 0xb8b3e (0x4e95 bytes) 2024-09-13T09:26:17.373 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:17.381 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:26:17.392 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:17.392 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:17.393 INFO:teuthology.orchestra.run.smithi063.stdout:1015 write 0xc65a9cd thru 0xc65e35f (0x3993 bytes) 2024-09-13T09:26:17.394 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:26:17.401 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:17.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:17.407 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:17.407 INFO:teuthology.orchestra.run.smithi063.stdout:1017 punch from 0xad43fa7 to 0xad49045, (0x509e bytes) 2024-09-13T09:26:17.407 INFO:teuthology.orchestra.run.smithi063.stdout:1018 write 0x1d31553 thru 0x1d3a343 (0x8df1 bytes) 2024-09-13T09:26:17.412 INFO:teuthology.orchestra.run.smithi063.stdout:1022 read 0x90d79e8 thru 0x90e7163 (0xf77c bytes) 2024-09-13T09:26:17.412 INFO:teuthology.orchestra.run.smithi063.stdout:1024 write 0x20b18cb thru 0x20b4a7f (0x31b5 bytes) 2024-09-13T09:26:17.415 INFO:teuthology.orchestra.run.smithi063.stdout:1025 write 0x8b9d6f8 thru 0x8ba7d86 (0xa68f bytes) 2024-09-13T09:26:17.425 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:17.428 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:17.434 INFO:teuthology.orchestra.run.smithi063.stdout:1027 punch from 0xc226258 to 0xc22c93f, (0x66e7 bytes) 2024-09-13T09:26:17.434 INFO:teuthology.orchestra.run.smithi063.stdout:1028 punch from 0x8be12d5 to 0x8bea59e, (0x92c9 bytes) 2024-09-13T09:26:17.434 INFO:teuthology.orchestra.run.smithi063.stdout:1029 write 0x136e07d thru 0x1371b39 (0x3abd bytes) 2024-09-13T09:26:17.439 INFO:teuthology.orchestra.run.smithi063.stdout:1030 read 0x57fb10b thru 0x57fd58a (0x2480 bytes) 2024-09-13T09:26:17.439 INFO:teuthology.orchestra.run.smithi063.stdout:1031 write 0x62d6e84 thru 0x62d7e84 (0x1001 bytes) 2024-09-13T09:26:17.448 INFO:teuthology.orchestra.run.smithi063.stdout:1033 trunc from 0xc65e360 to 0xc4bc3bd 2024-09-13T09:26:17.449 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:17.449+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:17.460 INFO:teuthology.orchestra.run.smithi063.stdout:1035 punch from 0x4016cb2 to 0x401fe94, (0x91e2 bytes) 2024-09-13T09:26:17.460 INFO:teuthology.orchestra.run.smithi063.stdout:1036 read 0x9667a8b thru 0x967410a (0xc680 bytes) 2024-09-13T09:26:17.460 INFO:teuthology.orchestra.run.smithi063.stdout:1037 punch from 0x6ace470 to 0x6ad42f6, (0x5e86 bytes) 2024-09-13T09:26:17.460 INFO:teuthology.orchestra.run.smithi063.stdout:1040 read 0x9da2933 thru 0x9da959c (0x6c6a bytes) 2024-09-13T09:26:17.461 INFO:teuthology.orchestra.run.smithi063.stdout:1041 write 0x5f009d5 thru 0x5f0b07f (0xa6ab bytes) 2024-09-13T09:26:17.481 INFO:teuthology.orchestra.run.smithi063.stdout:1042 punch from 0x8101e25 to 0x8109cb9, (0x7e94 bytes) 2024-09-13T09:26:17.481 INFO:teuthology.orchestra.run.smithi063.stdout:1043 punch from 0x718757a to 0x7195017, (0xda9d bytes) 2024-09-13T09:26:17.481 INFO:teuthology.orchestra.run.smithi063.stdout:1046 read 0x57ba68f thru 0x57be445 (0x3db7 bytes) 2024-09-13T09:26:17.482 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:17.487 INFO:teuthology.orchestra.run.smithi063.stdout:1048 trunc from 0xc4bc3bd to 0xed87e56 2024-09-13T09:26:17.496 INFO:teuthology.orchestra.run.smithi063.stdout:1050 punch from 0xac03b86 to 0xac0423e, (0x6b8 bytes) 2024-09-13T09:26:17.496 INFO:teuthology.orchestra.run.smithi063.stdout:1051 punch from 0x5418d4b to 0x541eaf8, (0x5dad bytes) 2024-09-13T09:26:17.496 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:17.496 INFO:teuthology.orchestra.run.smithi063.stdout:1054 writesame 0x45006d9 thru 0x45090a4 data_size 0x2(0x89cc bytes) 2024-09-13T09:26:17.507 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:17.503+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:17.516 INFO:teuthology.orchestra.run.smithi063.stdout:1055 write 0x624847 thru 0x62a57d (0x5d37 bytes) 2024-09-13T09:26:17.521 INFO:teuthology.orchestra.run.smithi063.stdout:1056 writesame 0xd6ea676 thru 0xd6f549d data_size 0x4(0xae28 bytes) 2024-09-13T09:26:17.541 INFO:teuthology.orchestra.run.smithi063.stdout:1057 writesame 0x95f2595 thru 0x95f312b data_size 0x3(0xb97 bytes) 2024-09-13T09:26:17.546 INFO:teuthology.orchestra.run.smithi063.stdout:1058 read 0xbbc3bc2 thru 0xbbd13d5 (0xd814 bytes) 2024-09-13T09:26:17.546 INFO:teuthology.orchestra.run.smithi063.stdout:1059 clone 11 order 20 su 262144 sc 8 2024-09-13T09:26:17.564 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:17.564 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:17.564 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:17.564 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:17.564 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:17.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:17.609 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:17.605+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:270 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.176307189999999, 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087584770000000006, 2024-09-13T09:26:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 270 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497697+0000", 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497696+0000", 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497707+0000", 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1214e-05 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497716+0000", 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0250000000000008e-06 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.497723+0000", 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5139999999999998e-06 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504669+0000", 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069460399999999997 2024-09-13T09:26:17.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504713+0000", 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3582999999999999e-05 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.504887+0000", 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174196 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.505909+0000", 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001022565 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506362+0000", 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045264300000000002 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506415+0000", 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2852000000000002e-05 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.506455+0000", 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0556000000000003e-05 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1851 3.1d 3:bdd77b2d:::rbd_data.1118e3e5e25d.000000000000004d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.590848157, 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084777419999999999, 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1851 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083156+0000", 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083154+0000", 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083170+0000", 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6062e-05 2024-09-13T09:26:17.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083198+0000", 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8191000000000002e-05 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083222+0000", 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083391+0000", 2024-09-13T09:26:17.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000169276 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083553+0000", 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016206400000000001 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.083765+0000", 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021140699999999999 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.085517+0000", 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001752691 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091479+0000", 2024-09-13T09:26:17.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059616670000000004 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091557+0000", 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8090999999999995e-05 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.091634+0000", 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6495999999999994e-05 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1976 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.570241496, 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093928829999999994, 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1976 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103763+0000", 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103762+0000", 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103767+0000", 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3550000000000004e-06 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103777+0000", 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5279999999999992e-06 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.103784+0000", 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5739999999999998e-06 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106569+0000", 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002785151 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106668+0000", 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8807000000000005e-05 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107170+0000", 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050205400000000004 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109619+0000", 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024487469999999998 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113077+0000", 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034580180000000002 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113120+0000", 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3112000000000003e-05 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113155+0000", 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5231999999999997e-05 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1977 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.569633660999999, 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088773199999999993, 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1977 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104370+0000", 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104369+0000", 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104390+0000", 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0999999999999999e-05 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104412+0000", 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2248000000000001e-05 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.104423+0000", 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0321e-05 2024-09-13T09:26:17.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107700+0000", 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032770329999999999 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.107828+0000", 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012850399999999999 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.108517+0000", 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068841499999999995 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111988+0000", 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034716550000000001 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113166+0000", 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001177893 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113215+0000", 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8758999999999999e-05 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113248+0000", 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2670999999999999e-05 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1979 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.702 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.567131588000001, 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085138350000000008, 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1979 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:17.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106872+0000", 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106871+0000", 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106880+0000", 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0760000000000003e-06 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106902+0000", 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2014e-05 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.106911+0000", 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7410000000000004e-06 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110772+0000", 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038607749999999999 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.110883+0000", 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011127400000000001 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.111390+0000", 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050661 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115162+0000", 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037724690000000001 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115242+0000", 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0073000000000004e-05 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115303+0000", 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0631000000000003e-05 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115386+0000", 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3181000000000002e-05 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:1981 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.564743592999999, 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083091610000000007, 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1981 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109260+0000", 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109259+0000", 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109267+0000", 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109280+0000", 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2862e-05 2024-09-13T09:26:17.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.109288+0000", 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7409999999999998e-06 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113375+0000", 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040869579999999999 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113466+0000", 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0756999999999999e-05 2024-09-13T09:26:17.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.113913+0000", 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00044651000000000001 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.115567+0000", 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016543949999999999 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117501+0000", 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001933702 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117544+0000", 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3624000000000001e-05 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.117570+0000", 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5347e-05 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2127 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.457826809, 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082727870000000002, 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2127 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216177+0000", 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216180+0000", 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.399e-06 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216188+0000", 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9699999999999999e-06 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216195+0000", 2024-09-13T09:26:17.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.725e-06 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224153+0000", 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079579420000000008 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224278+0000", 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012521799999999999 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224450+0000", 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000172032 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2128 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1081344~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.457803349000001, 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085956320000000006, 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2128 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216201+0000", 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216200+0000", 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216204+0000", 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6399999999999999e-06 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216212+0000", 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1270000000000003e-06 2024-09-13T09:26:17.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216219+0000", 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7399999999999998e-06 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224493+0000", 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0082740169999999998 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224636+0000", 2024-09-13T09:26:17.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014311700000000001 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224796+0000", 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016049 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2129 3.16 3:6a8ba6a5:::rbd_data.1118474c503.0000000000000001:6 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.457779263999999, 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090836349999999996, 2024-09-13T09:26:17.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2129 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216225+0000", 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216224+0000", 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216228+0000", 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.428e-06 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216235+0000", 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.216242+0000", 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.973e-06 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.224846+0000", 2024-09-13T09:26:17.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086038030000000001 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225053+0000", 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020684800000000001 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.225308+0000", 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025549900000000001 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2136 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.447627033, 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086734050000000004, 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2136 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226377+0000", 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226376+0000", 2024-09-13T09:26:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226386+0000", 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0789e-05 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226416+0000", 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9660000000000001e-05 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226437+0000", 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0982999999999999e-05 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.229652+0000", 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032148680000000001 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.229719+0000", 2024-09-13T09:26:17.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7087000000000003e-05 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.230441+0000", 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072155899999999998 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232323+0000", 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018823069999999999 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234949+0000", 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002625966 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235010+0000", 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1101999999999999e-05 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235050+0000", 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0454999999999998e-05 2024-09-13T09:26:17.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2137 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.447557906, 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095282279999999997, 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2137 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226446+0000", 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226450+0000", 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226463+0000", 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3338e-05 2024-09-13T09:26:17.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226467+0000", 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.481e-06 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231090+0000", 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046230450000000001 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231197+0000", 2024-09-13T09:26:17.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106604 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.231659+0000", 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000462189 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234788+0000", 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031288269999999998 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235890+0000", 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001101929 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235929+0000", 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8714000000000002e-05 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.235974+0000", 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5611000000000001e-05 2024-09-13T09:26:17.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2138 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.447533135, 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012214620000000001, 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2138 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226471+0000", 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226474+0000", 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8670000000000004e-06 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226485+0000", 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0359e-05 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226501+0000", 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6671999999999999e-05 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232383+0000", 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058812739999999997 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232455+0000", 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2107999999999995e-05 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.232795+0000", 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000340532 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234797+0000", 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020017889999999999 2024-09-13T09:26:17.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238578+0000", 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003780533 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238644+0000", 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5840999999999995e-05 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238686+0000", 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1984000000000003e-05 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2139 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.447492957, 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012363097999999999, 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2139 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:17.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226511+0000", 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226514+0000", 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.608e-06 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226519+0000", 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.226523+0000", 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9940000000000002e-06 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233419+0000", 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006896297 2024-09-13T09:26:17.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.233548+0000", 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012899999999999999 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.234129+0000", 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058001899999999998 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.237501+0000", 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033728790000000001 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238760+0000", 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001258388 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238831+0000", 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0967e-05 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.238874+0000", 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3439999999999997e-05 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.209281191000001, 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:17.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.184938928999999, 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:17.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:17.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.180570566, 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.817141908, 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:17.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:17.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.980831427, 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:17.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:5801 3.5 e37/21 ondisk, result = 0)", 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.727147+0000", 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9468574150000002, 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012834019, 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727147+0000", 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727147+0000", 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727145+0000", 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727153+0000", 2024-09-13T09:26:17.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4989999999999997e-06 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727170+0000", 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6640999999999999e-05 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.727182+0000", 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1969e-05 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739763+0000", 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012580996000000001 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739781+0000", 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8603000000000001e-05 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739981+0000", 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019951800000000001 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7278253990000003, 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:17.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:17.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:17.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.790 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:17.790 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:17.790 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:17.791 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:17.792 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:17.789+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:17.892 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:17.887+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:17.947 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:17.947 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:17.947 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:17.948 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:17.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:17.993 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:17.993+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:18.066 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:18.066 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:18.066 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:18.067 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:18.067 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:18.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:18.117 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:18.117+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:18.196 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:18.191+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:18.231 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:18.231 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:18.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:18.231 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:18.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.667241809, 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:18.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:18.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21942 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.415536+0000", 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.81083619500000004, 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012825555000000001, 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21942 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415536+0000", 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415536+0000", 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415536+0000", 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415539+0000", 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.847e-06 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415542+0000", 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415546+0000", 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2629999999999999e-06 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.421383+0000", 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058373640000000003 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.421466+0000", 2024-09-13T09:26:18.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3059999999999994e-05 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.421766+0000", 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030011600000000001 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.427933+0000", 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061672569999999998 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428261+0000", 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032800899999999998 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428314+0000", 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2608999999999997e-05 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428362+0000", 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7493000000000001e-05 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21943 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.415551+0000", 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.810821448, 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013032108000000001, 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21943 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415551+0000", 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415551+0000", 2024-09-13T09:26:18.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415550+0000", 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415553+0000", 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4959999999999999e-06 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415557+0000", 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.048e-06 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415560+0000", 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5319999999999998e-06 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.422198+0000", 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066379129999999996 2024-09-13T09:26:18.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.422303+0000", 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104342 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.422680+0000", 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037741899999999999 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.427962+0000", 2024-09-13T09:26:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005282269 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428447+0000", 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048460199999999999 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428526+0000", 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9170000000000003e-05 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428583+0000", 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6644000000000002e-05 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21944 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.415572+0000", 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.81080054999999995, 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013137387, 2024-09-13T09:26:18.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21944 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415572+0000", 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415572+0000", 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415571+0000", 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415574+0000", 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8329999999999998e-06 2024-09-13T09:26:18.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415582+0000", 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7419999999999997e-06 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415587+0000", 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9259999999999999e-06 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.422982+0000", 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0073948679999999998 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.423041+0000", 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9831999999999999e-05 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.423261+0000", 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000219698 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.427979+0000", 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004718056 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428636+0000", 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00065687099999999995 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428670+0000", 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4028e-05 2024-09-13T09:26:18.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428709+0000", 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8874000000000001e-05 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21945 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.415610+0000", 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.81076231099999996, 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013258696, 2024-09-13T09:26:18.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21945 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415610+0000", 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415610+0000", 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415609+0000", 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415612+0000", 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.033e-06 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415631+0000", 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8320000000000001e-05 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415637+0000", 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6610000000000003e-06 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.423652+0000", 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0080145730000000005 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.423706+0000", 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4304e-05 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.423959+0000", 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000253057 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428379+0000", 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004419114 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428761+0000", 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000382047 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428822+0000", 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1805999999999997e-05 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428869+0000", 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6161000000000001e-05 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21946 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.415648+0000", 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.81072455600000004, 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015546506999999999, 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21946 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415648+0000", 2024-09-13T09:26:18.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415648+0000", 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415647+0000", 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415650+0000", 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2509999999999999e-06 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415663+0000", 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.204e-05 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.415671+0000", 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9570000000000008e-06 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.424254+0000", 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085826900000000005 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.424305+0000", 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0665999999999999e-05 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.424604+0000", 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029948799999999999 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.428385+0000", 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037804079999999999 2024-09-13T09:26:18.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.431082+0000", 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002697175 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.431149+0000", 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7420000000000002e-05 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.431194+0000", 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4776e-05 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21983 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.460838+0000", 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.765534571, 2024-09-13T09:26:18.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013125217999999999, 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21983 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.270 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460838+0000", 2024-09-13T09:26:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460838+0000", 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460837+0000", 2024-09-13T09:26:18.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460842+0000", 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2739999999999998e-06 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460856+0000", 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.33e-05 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460868+0000", 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2761e-05 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.465846+0000", 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049775949999999996 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.465931+0000", 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5382000000000003e-05 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.466191+0000", 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025993799999999998 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.469520+0000", 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033286600000000002 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.473905+0000", 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043846570000000001 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.473936+0000", 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1937999999999998e-05 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.473963+0000", 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6335000000000001e-05 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21984 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.460876+0000", 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76549608800000002, 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013154575999999999, 2024-09-13T09:26:18.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21984 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460876+0000", 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460876+0000", 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460876+0000", 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460909+0000", 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3152999999999999e-05 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460922+0000", 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3008e-05 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460936+0000", 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4214e-05 2024-09-13T09:26:18.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.467010+0000", 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060737009999999999 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.467071+0000", 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1799999999999998e-05 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.467316+0000", 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024509400000000001 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.472502+0000", 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051856460000000004 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.473972+0000", 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014699769999999999 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.474005+0000", 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2431000000000002e-05 2024-09-13T09:26:18.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.474031+0000", 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6081999999999999e-05 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21985 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.460945+0000", 2024-09-13T09:26:18.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.765427419, 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0149293, 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21985 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460945+0000", 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460945+0000", 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460944+0000", 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460949+0000", 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2730000000000002e-06 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460970+0000", 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0407999999999998e-05 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460986+0000", 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6226e-05 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.467813+0000", 2024-09-13T09:26:18.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006826548 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.467909+0000", 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6776999999999999e-05 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.468262+0000", 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035312899999999999 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475732+0000", 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074699700000000003 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475763+0000", 2024-09-13T09:26:18.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0142999999999999e-05 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475836+0000", 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3378000000000001e-05 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475874+0000", 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21986 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.460995+0000", 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76537725099999998, 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015187233, 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21986 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460995+0000", 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460995+0000", 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.460994+0000", 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461000+0000", 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:26:18.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461017+0000", 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7384000000000002e-05 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461027+0000", 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0383999999999999e-05 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.468743+0000", 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077160359999999999 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.468808+0000", 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4618999999999999e-05 2024-09-13T09:26:18.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.469081+0000", 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027347199999999997 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475931+0000", 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068500849999999997 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475996+0000", 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4395999999999998e-05 2024-09-13T09:26:18.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476039+0000", 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3152999999999998e-05 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476182+0000", 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014307999999999999 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21987 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.461035+0000", 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76533730300000002, 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015361079999999999, 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21987 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461035+0000", 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461035+0000", 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461034+0000", 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461040+0000", 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7910000000000001e-06 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461058+0000", 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8355999999999998e-05 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461074+0000", 2024-09-13T09:26:18.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6098000000000001e-05 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.469537+0000", 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084627350000000007 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.469594+0000", 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6350000000000001e-05 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.469866+0000", 2024-09-13T09:26:18.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000272451 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475943+0000", 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060771189999999998 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476273+0000", 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033014399999999999 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476359+0000", 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5414000000000006e-05 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476396+0000", 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7268000000000003e-05 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21988 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.461083+0000", 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76528914800000003, 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015442717, 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21988 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461083+0000", 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461083+0000", 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461082+0000", 2024-09-13T09:26:18.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461088+0000", 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461110+0000", 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2427999999999999e-05 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461129+0000", 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9065000000000001e-05 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.470531+0000", 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009401398 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.470638+0000", 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107025 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.470891+0000", 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025329399999999999 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475955+0000", 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050639159999999999 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476433+0000", 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00047767600000000001 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476505+0000", 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2179000000000001e-05 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476526+0000", 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0988999999999998e-05 2024-09-13T09:26:18.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21989 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.461140+0000", 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.765232358, 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015481485, 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21989 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461140+0000", 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461140+0000", 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461139+0000", 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461144+0000", 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2340000000000002e-06 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461163+0000", 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8898000000000001e-05 2024-09-13T09:26:18.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461172+0000", 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.471431+0000", 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010259074999999999 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.471523+0000", 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2633999999999994e-05 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.471900+0000", 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00037689199999999998 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.475963+0000", 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040630950000000001 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476561+0000", 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00059737199999999999 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476586+0000", 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4888999999999998e-05 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476621+0000", 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5649999999999999e-05 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21990 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.461179+0000", 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76519319900000005, 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015530657, 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21990 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461179+0000", 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461179+0000", 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461178+0000", 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461183+0000", 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.006e-06 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461210+0000", 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6174e-05 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461218+0000", 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6060000000000003e-06 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.472572+0000", 2024-09-13T09:26:18.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011354061 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.472691+0000", 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118595 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.473317+0000", 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00062627400000000001 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476200+0000", 2024-09-13T09:26:18.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002883282 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476657+0000", 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045666599999999999 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476682+0000", 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4484000000000001e-05 2024-09-13T09:26:18.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476710+0000", 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8089000000000002e-05 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:21991 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.461226+0000", 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.76514626399999996, 2024-09-13T09:26:18.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01559812, 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 21991 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461226+0000", 2024-09-13T09:26:18.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461226+0000", 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461225+0000", 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461230+0000", 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2120000000000002e-06 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461247+0000", 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6900000000000001e-05 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.461256+0000", 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6680000000000007e-06 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.474084+0000", 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012828367 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.474193+0000", 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108121 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.474576+0000", 2024-09-13T09:26:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038350400000000001 2024-09-13T09:26:18.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476633+0000", 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020567300000000001 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476742+0000", 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109601 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476788+0000", 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5253000000000002e-05 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.476824+0000", 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6375999999999999e-05 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22022 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.497277+0000", 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72909558799999996, 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011961869999999999, 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22022 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.497277+0000", 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.497277+0000", 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.497276+0000", 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.497285+0000", 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.081e-06 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.497300+0000", 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.558e-05 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.497309+0000", 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.501507+0000", 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041972429999999998 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.501623+0000", 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.501912+0000", 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028891799999999999 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.509063+0000", 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071510699999999998 2024-09-13T09:26:18.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.509121+0000", 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7677999999999999e-05 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.509181+0000", 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0213000000000001e-05 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.509238+0000", 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7565999999999997e-05 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22053 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.521993+0000", 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.70437924600000001, 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014873391999999999, 2024-09-13T09:26:18.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22053 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.521993+0000", 2024-09-13T09:26:18.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.521993+0000", 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.521992+0000", 2024-09-13T09:26:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.521996+0000", 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2739999999999999e-06 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522007+0000", 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1759999999999999e-05 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522012+0000", 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309999999999997e-06 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.531675+0000", 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096632109999999997 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.531793+0000", 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011783899999999999 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.532301+0000", 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050801199999999996 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.535547+0000", 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003246147 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.536726+0000", 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011790570000000001 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.536793+0000", 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6728000000000002e-05 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.536866+0000", 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3499999999999998e-05 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22054 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.522023+0000", 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.70434927400000003, 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015077102, 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22054 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522023+0000", 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522023+0000", 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522023+0000", 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522025+0000", 2024-09-13T09:26:18.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8490000000000002e-06 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522030+0000", 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522033+0000", 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.377e-06 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.533123+0000", 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011089259000000001 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.533193+0000", 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0015e-05 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.533584+0000", 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039156999999999999 2024-09-13T09:26:18.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.536898+0000", 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033133590000000001 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.536964+0000", 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6682999999999998e-05 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.537036+0000", 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2121999999999998e-05 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.537100+0000", 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3491999999999995e-05 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22055 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.522077+0000", 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.70429520599999995, 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015978284999999998, 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22055 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522077+0000", 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522077+0000", 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522076+0000", 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522080+0000", 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3749999999999999e-06 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522085+0000", 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2900000000000002e-06 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522089+0000", 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4510000000000001e-06 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.534086+0000", 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011997348999999999 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.534491+0000", 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040467999999999998 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.534912+0000", 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00042112600000000002 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.537146+0000", 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022345440000000002 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.537933+0000", 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078676600000000005 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.538019+0000", 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6290000000000002e-05 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.538055+0000", 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5868000000000003e-05 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22056 3.10 3:0aec1d35:::rbd_object_map.1118a28cd5c4:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:17.522159+0000", 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.70421335500000004, 2024-09-13T09:26:18.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016200856999999999, 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22056 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522159+0000", 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522159+0000", 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522158+0000", 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522162+0000", 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6169999999999998e-06 2024-09-13T09:26:18.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522168+0000", 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2539999999999998e-06 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.522172+0000", 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6239999999999999e-06 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.535611+0000", 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013439328 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.535848+0000", 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023683100000000001 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.536252+0000", 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000404041 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.538070+0000", 2024-09-13T09:26:18.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018182770000000001 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.538149+0000", 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9133000000000003e-05 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.538241+0000", 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2094000000000003e-05 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:17.538360+0000", 2024-09-13T09:26:18.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000118158 2024-09-13T09:26:18.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.375 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.375 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.375 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:18.377 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:18.377 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:18.373+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:18.392 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:26:18.395 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:26:18.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:26:18.477 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:18.471+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:18.492 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:18.493 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:18.493 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:18.493 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:18.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:18.578 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:18.577+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:18.605 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:18.605 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:18.606 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:18.606 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:18.606 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:18.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:18.675 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T09:26:18.680 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:18.675+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:18.688 INFO:teuthology.orchestra.run.smithi063.stdout:1060 read 0xdd05343 thru 0xdd0f464 (0xa122 bytes) 2024-09-13T09:26:18.694 INFO:teuthology.orchestra.run.smithi063.stdout:1061 write 0x4127cd4 thru 0x412af7f (0x32ac bytes) 2024-09-13T09:26:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:1064 write 0xbe829fa thru 0xbe8e5d6 (0xbbdd bytes) 2024-09-13T09:26:18.712 INFO:teuthology.orchestra.run.smithi063.stdout:1065 write 0x24dfe4a thru 0x24e4354 (0x450b bytes) 2024-09-13T09:26:18.721 INFO:teuthology.orchestra.run.smithi063.stdout:1069 write 0xb7beb73 thru 0xb7cae42 (0xc2d0 bytes) 2024-09-13T09:26:18.726 INFO:teuthology.orchestra.run.smithi063.stdout:1071 writesame 0xbf777c9 thru 0xbf82efb data_size 0x51(0xb733 bytes) 2024-09-13T09:26:18.744 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.779981249999999, 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:18.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:18.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:18.750 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.783591795, 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:18.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.375018163, 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014920113, 2024-09-13T09:26:18.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365977+0000", 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365987+0000", 2024-09-13T09:26:18.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366004+0000", 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.662e-05 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366022+0000", 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7883000000000001e-05 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376091+0000", 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010069171999999999 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376246+0000", 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000155377 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376548+0000", 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030137300000000002 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380798+0000", 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004250406 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380813+0000", 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4357e-05 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380862+0000", 2024-09-13T09:26:18.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9137000000000001e-05 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380898+0000", 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6158999999999997e-05 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:18.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.374959584999999, 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016811104, 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366035+0000", 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366045+0000", 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0327e-05 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366062+0000", 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6483000000000001e-05 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366074+0000", 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377874+0000", 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011799983999999999 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378006+0000", 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013130100000000001 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378143+0000", 2024-09-13T09:26:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137157 2024-09-13T09:26:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380914+0000", 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002771281 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382760+0000", 2024-09-13T09:26:18.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001846253 2024-09-13T09:26:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382810+0000", 2024-09-13T09:26:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9483e-05 2024-09-13T09:26:18.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382847+0000", 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7749000000000001e-05 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.374879882999998, 2024-09-13T09:26:18.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018719919000000002, 2024-09-13T09:26:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:18.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366115+0000", 2024-09-13T09:26:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366313+0000", 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000198156 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366331+0000", 2024-09-13T09:26:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7739000000000001e-05 2024-09-13T09:26:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366345+0000", 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4226000000000001e-05 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378684+0000", 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012339325 2024-09-13T09:26:18.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378827+0000", 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014284700000000001 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.379116+0000", 2024-09-13T09:26:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028930399999999999 2024-09-13T09:26:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382675+0000", 2024-09-13T09:26:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035590050000000001 2024-09-13T09:26:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384568+0000", 2024-09-13T09:26:18.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001892707 2024-09-13T09:26:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384706+0000", 2024-09-13T09:26:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013804099999999999 2024-09-13T09:26:18.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384836+0000", 2024-09-13T09:26:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012977900000000001 2024-09-13T09:26:18.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.374633725000002, 2024-09-13T09:26:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01866932, 2024-09-13T09:26:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-13T09:26:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366361+0000", 2024-09-13T09:26:18.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366372+0000", 2024-09-13T09:26:18.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0857e-05 2024-09-13T09:26:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366388+0000", 2024-09-13T09:26:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:26:18.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366401+0000", 2024-09-13T09:26:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.326e-05 2024-09-13T09:26:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380005+0000", 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013604036 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380100+0000", 2024-09-13T09:26:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4468000000000004e-05 2024-09-13T09:26:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380211+0000", 2024-09-13T09:26:18.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011084199999999999 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383662+0000", 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034510460000000001 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384905+0000", 2024-09-13T09:26:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001243009 2024-09-13T09:26:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384996+0000", 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385032+0000", 2024-09-13T09:26:18.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5528000000000003e-05 2024-09-13T09:26:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.374580876000003, 2024-09-13T09:26:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020964449999999999, 2024-09-13T09:26:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366414+0000", 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366423+0000", 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366443+0000", 2024-09-13T09:26:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:26:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366455+0000", 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380966+0000", 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014510662000000001 2024-09-13T09:26:18.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381062+0000", 2024-09-13T09:26:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6526000000000004e-05 2024-09-13T09:26:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381259+0000", 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019658499999999999 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383675+0000", 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024160480000000001 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387295+0000", 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036198020000000001 2024-09-13T09:26:18.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387342+0000", 2024-09-13T09:26:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7299000000000002e-05 2024-09-13T09:26:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387380+0000", 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7379000000000002e-05 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.374156849000002, 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020678247, 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366838+0000", 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367071+0000", 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023322800000000001 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367094+0000", 2024-09-13T09:26:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3079999999999999e-05 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367111+0000", 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7008999999999999e-05 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381788+0000", 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014676850999999999 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381894+0000", 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010610799999999999 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382045+0000", 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015073699999999999 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383689+0000", 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016436630000000001 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387445+0000", 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037564 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387488+0000", 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3402000000000002e-05 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387517+0000", 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8975999999999998e-05 2024-09-13T09:26:18.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.373867162000003, 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020557408999999999, 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367128+0000", 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367138+0000", 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0672999999999999e-05 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367157+0000", 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8779000000000001e-05 2024-09-13T09:26:18.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367174+0000", 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6464e-05 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382926+0000", 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015752759000000002 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383077+0000", 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150965 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383228+0000", 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015101399999999999 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384458+0000", 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012294370000000001 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387599+0000", 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031411960000000002 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387647+0000", 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7648999999999997e-05 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387686+0000", 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9653000000000001e-05 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.822 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.373808056999998, 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022933500999999998, 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.823 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367187+0000", 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367208+0000", 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1793e-05 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367225+0000", 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668999999999999e-05 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367239+0000", 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4005999999999999e-05 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383748+0000", 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016508719000000002 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383933+0000", 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018481500000000001 2024-09-13T09:26:18.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384122+0000", 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189626 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385049+0000", 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092678599999999997 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390003+0000", 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049541819999999997 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390075+0000", 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2094999999999993e-05 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390121+0000", 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6082999999999999e-05 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783752677000003, 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015204896000000001, 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 135 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957242+0000", 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957250+0000", 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957268+0000", 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7459999999999999e-05 2024-09-13T09:26:18.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957277+0000", 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3559999999999999e-06 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965456+0000", 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792359999999995 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965567+0000", 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110877 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965741+0000", 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017349600000000001 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967304+0000", 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001563444 2024-09-13T09:26:18.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972380+0000", 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050761019999999999 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972425+0000", 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4366000000000002e-05 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972448+0000", 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3455000000000002e-05 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783708607000001, 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015208101, 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 136 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957297+0000", 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9909999999999997e-06 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957339+0000", 2024-09-13T09:26:18.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2267e-05 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957371+0000", 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2518000000000003e-05 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966255+0000", 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088832679999999997 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966353+0000", 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8658000000000003e-05 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966486+0000", 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013309500000000001 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969257+0000", 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027707209999999999 2024-09-13T09:26:18.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972453+0000", 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031962789999999998 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972478+0000", 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4655e-05 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972495+0000", 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7407000000000001e-05 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783593746000001, 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015156871000000001, 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957401+0000", 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957509+0000", 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010808 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957536+0000", 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6237e-05 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957542+0000", 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2020000000000001e-06 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967352+0000", 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098099580000000006 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967447+0000", 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5375000000000001e-05 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967584+0000", 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136463 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971541+0000", 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039574930000000003 2024-09-13T09:26:18.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972499+0000", 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095777799999999995 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972527+0000", 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7900999999999999e-05 2024-09-13T09:26:18.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972559+0000", 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2191999999999999e-05 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783443908999999, 2024-09-13T09:26:18.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015104328, 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957551+0000", 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957556+0000", 2024-09-13T09:26:18.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.318e-06 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957565+0000", 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.115e-06 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957572+0000", 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968300+0000", 2024-09-13T09:26:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010727617 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968387+0000", 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7021000000000006e-05 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968482+0000", 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5081999999999995e-05 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971595+0000", 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031126639999999998 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972588+0000", 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099296600000000007 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972635+0000", 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7136999999999999e-05 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972656+0000", 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1480000000000001e-05 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783415108, 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015429492, 2024-09-13T09:26:18.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957580+0000", 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957586+0000", 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:26:18.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957620+0000", 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3215e-05 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957651+0000", 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1027e-05 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969570+0000", 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011919511000000001 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969674+0000", 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104248 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969784+0000", 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010937700000000001 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972662+0000", 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028781589999999999 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972925+0000", 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026289899999999998 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972976+0000", 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0702999999999999e-05 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973010+0000", 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4743000000000003e-05 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783310503999999, 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020098000000000001, 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:18.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957793+0000", 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108837 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957808+0000", 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957814+0000", 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970251+0000", 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012436523999999999 2024-09-13T09:26:18.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970350+0000", 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9600999999999997e-05 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970506+0000", 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015539500000000001 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972667+0000", 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002161619 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977681+0000", 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050135930000000002 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977722+0000", 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1634999999999997e-05 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977783+0000", 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0968000000000003e-05 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.783174377000002, 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022700502000000001, 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:18.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957821+0000", 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957825+0000", 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957843+0000", 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7492000000000001e-05 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957851+0000", 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971666+0000", 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013814707000000001 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971841+0000", 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174659 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971976+0000", 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013485600000000001 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973161+0000", 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011852130000000001 2024-09-13T09:26:18.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980427+0000", 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072666440000000001 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980487+0000", 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9373000000000002e-05 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980522+0000", 2024-09-13T09:26:18.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5244999999999999e-05 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:4479 3.d 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9052358300000005, 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051456631000000003, 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4479 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:18.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835751+0000", 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999909 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854271+0000", 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01851972 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854305+0000", 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4258000000000003e-05 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854317+0000", 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2013e-05 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854487+0000", 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017018 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854768+0000", 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028067700000000001 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.855666+0000", 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000898404 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.881583+0000", 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025916947999999999 2024-09-13T09:26:18.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887097+0000", 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055134479999999998 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887161+0000", 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4077000000000003e-05 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887217+0000", 2024-09-13T09:26:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6128000000000002e-05 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6281 3.13 3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8939690249999996, 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024029966, 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6281 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847025+0000", 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847238+0000", 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021285799999999999 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847259+0000", 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0908e-05 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847267+0000", 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.67e-06 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847329+0000", 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2016999999999999e-05 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847464+0000", 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134497 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.853158+0000", 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056936219999999997 2024-09-13T09:26:18.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870613+0000", 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017455404000000001 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870743+0000", 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012992099999999999 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870911+0000", 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168276 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.871057+0000", 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014575999999999999 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6591 3.13 3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:18.885 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6054664169999997, 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015116329, 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6591 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135528+0000", 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135537+0000", 2024-09-13T09:26:18.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0669999999999998e-06 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135554+0000", 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135563+0000", 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5380000000000008e-06 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:18.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135634+0000", 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1106999999999995e-05 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135759+0000", 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012423999999999999 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.139009+0000", 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032509520000000001 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150356+0000", 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011346962 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150461+0000", 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104998 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150550+0000", 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8622999999999995e-05 2024-09-13T09:26:18.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150646+0000", 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5822000000000005e-05 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:18.891 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:18.893 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1072 trunc from 0xed87e56 to 0x68514bf 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1073 read 0x616a12 thru 0x619294 (0x2883 bytes) 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1075 writesame 0x8dba417 thru 0x8dc3681 data_size 0x1(0x926b bytes) 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1076 trunc from 0x8dc3682 to 0x853b14b 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1077 write 0x9ee494a thru 0x9eeb69d (0x6d54 bytes) 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1078 punch from 0x2fb5ecf to 0x2fc4884, (0xe9b5 bytes) 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:1080 read 0x4dccb77 thru 0x4dda712 (0xdb9c bytes) 2024-09-13T09:26:18.895 INFO:teuthology.orchestra.run.smithi063.stdout:1081 write 0xba5be8d thru 0xba69668 (0xd7dc bytes) 2024-09-13T09:26:18.895 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:18.887+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:18.895 INFO:teuthology.orchestra.run.smithi063.stdout:1082 write 0xdd27406 thru 0xdd2d3e0 (0x5fdb bytes) 2024-09-13T09:26:18.900 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:18.900 INFO:teuthology.orchestra.run.smithi063.stdout:1086 read 0xbd9da52 thru 0xbda97b6 (0xbd65 bytes) 2024-09-13T09:26:18.901 INFO:teuthology.orchestra.run.smithi063.stdout:1087 writesame 0xc65abe5 thru 0xc6668fd data_size 0x1(0xbd19 bytes) 2024-09-13T09:26:18.939 INFO:teuthology.orchestra.run.smithi063.stdout:1088 writesame 0x2a3e33 thru 0x2ae932 data_size 0x4(0xab00 bytes) 2024-09-13T09:26:18.965 INFO:teuthology.orchestra.run.smithi063.stdout:1089 writesame 0xd0e37f0 thru 0xd0ecfcc data_size 0x1(0x97dd bytes) 2024-09-13T09:26:18.990 INFO:teuthology.orchestra.run.smithi063.stdout:1090 punch from 0xc5256a9 to 0xc5351cd, (0xfb24 bytes) 2024-09-13T09:26:18.990 INFO:teuthology.orchestra.run.smithi063.stdout:1091 read 0x1624b66 thru 0x16331da (0xe675 bytes) 2024-09-13T09:26:18.993 INFO:teuthology.orchestra.run.smithi063.stdout:1092 read 0x22d7afb thru 0x22e684e (0xed54 bytes) 2024-09-13T09:26:18.994 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:18.991+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:19.001 INFO:teuthology.orchestra.run.smithi063.stdout:1093 read 0x838b72a thru 0x838f6a1 (0x3f78 bytes) 2024-09-13T09:26:19.012 INFO:teuthology.orchestra.run.smithi063.stdout:1095 trunc from 0xdd2d3e1 to 0x834d15e 2024-09-13T09:26:19.025 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:19.025 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:19.025 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:19.025 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:19.037 INFO:teuthology.orchestra.run.smithi063.stdout:1097 trunc from 0x834d15e to 0xc80d0d8 2024-09-13T09:26:19.041 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:19.065 INFO:teuthology.orchestra.run.smithi063.stdout:1101 write 0xec715a5 thru 0xec7ad1f (0x977b bytes) 2024-09-13T09:26:19.072 INFO:teuthology.orchestra.run.smithi063.stdout:1102 write 0x8dc7dcc thru 0x8dd3a64 (0xbc99 bytes) 2024-09-13T09:26:19.079 INFO:teuthology.orchestra.run.smithi063.stdout:1103 writesame 0x6ca1b21 thru 0x6ca2d0e data_size 0x22(0x11ee bytes) 2024-09-13T09:26:19.086 INFO:teuthology.orchestra.run.smithi063.stdout:1105 write 0x7fd435f thru 0x7fdc87a (0x851c bytes) 2024-09-13T09:26:19.094 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:19.091+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:19.108 INFO:teuthology.orchestra.run.smithi063.stdout:1107 trunc from 0xec7ad20 to 0x6c190cb 2024-09-13T09:26:19.142 INFO:teuthology.orchestra.run.smithi063.stdout:1108 trunc from 0x6c190cb to 0x1820aa4 2024-09-13T09:26:19.185 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:19.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:19.185 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:19.185 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:19.185 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:19.195 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:19.193+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:19.197 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:19.200 INFO:teuthology.orchestra.run.smithi063.stdout:1109 write 0xcec1619 thru 0xcecefda (0xd9c2 bytes) 2024-09-13T09:26:19.207 INFO:teuthology.orchestra.run.smithi063.stdout:1110 write 0x6373d93 thru 0x63781b3 (0x4421 bytes) 2024-09-13T09:26:19.214 INFO:teuthology.orchestra.run.smithi063.stdout:1112 write 0xb5ea48f thru 0xb5ed1db (0x2d4d bytes) 2024-09-13T09:26:19.220 INFO:teuthology.orchestra.run.smithi063.stdout:1113 read 0x3286ed6 thru 0x328ddd0 (0x6efb bytes) 2024-09-13T09:26:19.220 INFO:teuthology.orchestra.run.smithi063.stdout:1114 writesame 0x4c1c65f thru 0x4c1d7a5 data_size 0x1(0x1147 bytes) 2024-09-13T09:26:19.229 INFO:teuthology.orchestra.run.smithi063.stdout:1116 writesame 0xb5f0c64 thru 0xb5fd91e data_size 0x1(0xccbb bytes) 2024-09-13T09:26:19.263 INFO:teuthology.orchestra.run.smithi063.stdout:1117 punch from 0x73705d5 to 0x7374ca7, (0x46d2 bytes) 2024-09-13T09:26:19.263 INFO:teuthology.orchestra.run.smithi063.stdout:1119 read 0x1485fd0 thru 0x1491d1e (0xbd4f bytes) 2024-09-13T09:26:19.267 INFO:teuthology.orchestra.run.smithi063.stdout:1122 read 0x496921f thru 0x4970ea9 (0x7c8b bytes) 2024-09-13T09:26:19.268 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.268 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.268 INFO:teuthology.orchestra.run.smithi063.stdout:1125 read 0x7c813dc thru 0x7c85a46 (0x466b bytes) 2024-09-13T09:26:19.268 INFO:teuthology.orchestra.run.smithi063.stdout:1127 read 0xbd989a thru 0xbdd5d7 (0x3d3e bytes) 2024-09-13T09:26:19.274 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.274 INFO:teuthology.orchestra.run.smithi063.stdout:1129 writesame 0x229b28a thru 0x229b488 data_size 0x1(0x1ff bytes) 2024-09-13T09:26:19.283 INFO:teuthology.orchestra.run.smithi063.stdout:1130 read 0x8f13d3e thru 0x8f1a3a7 (0x666a bytes) 2024-09-13T09:26:19.283 INFO:teuthology.orchestra.run.smithi063.stdout:1131 write 0x1373323 thru 0x137cf3e (0x9c1c bytes) 2024-09-13T09:26:19.297 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:19.291+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 35.354265990999998, 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:19.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.337707778000002, 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.10296052, 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.100751382999999, 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.096406858000002, 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.079855879, 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.079267870999999, 2024-09-13T09:26:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.071561715000001, 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:19.346 INFO:teuthology.orchestra.run.smithi063.stdout:1132 trunc from 0xcecefdb to 0x2c60c8 2024-09-13T09:26:19.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:19.367 INFO:teuthology.orchestra.run.smithi063.stdout:1134 punch from 0x25d19a to 0x267523, (0xa389 bytes) 2024-09-13T09:26:19.367 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.367 INFO:teuthology.orchestra.run.smithi063.stdout:1137 read 0x10a495 thru 0x10b091 (0xbfd bytes) 2024-09-13T09:26:19.397 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:19.397+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:26:19.398 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:26:19.401 INFO:teuthology.orchestra.run.smithi063.stdout:1139 write 0x87d0c48 thru 0x87d10a0 (0x459 bytes) 2024-09-13T09:26:19.410 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:26:19.410 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:26:19.413 INFO:teuthology.orchestra.run.smithi063.stdout:1140 read 0x2bd007f thru 0x2bd01ea (0x16c bytes) 2024-09-13T09:26:19.413 INFO:teuthology.orchestra.run.smithi063.stdout:1141 read 0x7f8ae9 thru 0x801644 (0x8b5c bytes) 2024-09-13T09:26:19.413 INFO:teuthology.orchestra.run.smithi063.stdout:1142 read 0x4c2126a thru 0x4c26a3c (0x57d3 bytes) 2024-09-13T09:26:19.414 INFO:teuthology.orchestra.run.smithi063.stdout:1143 read 0x395241a thru 0x395c2d9 (0x9ec0 bytes) 2024-09-13T09:26:19.414 INFO:teuthology.orchestra.run.smithi063.stdout:1145 write 0x342608e thru 0x3426bd5 (0xb48 bytes) 2024-09-13T09:26:19.431 INFO:teuthology.orchestra.run.smithi063.stdout:1147 trunc from 0x87d10a1 to 0x1179ec8 2024-09-13T09:26:19.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:19.484 INFO:teuthology.orchestra.run.smithi063.stdout:1148 write 0xd176e60 thru 0xd17c01c (0x51bd bytes) 2024-09-13T09:26:19.486 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:19.486 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:19.487 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:19.487 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:19.489 INFO:teuthology.orchestra.run.smithi063.stdout:1149 writesame 0x58506b7 thru 0x585629e data_size 0x8(0x5be8 bytes) 2024-09-13T09:26:19.498 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:19.500 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:19.497+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:19.500 INFO:teuthology.orchestra.run.smithi063.stdout:1150 write 0x224f9fb thru 0x225f091 (0xf697 bytes) 2024-09-13T09:26:19.505 INFO:teuthology.orchestra.run.smithi063.stdout:1151 write 0xb55026b thru 0xb55907a (0x8e10 bytes) 2024-09-13T09:26:19.512 INFO:teuthology.orchestra.run.smithi063.stdout:1152 read 0x37d153d thru 0x37d8935 (0x73f9 bytes) 2024-09-13T09:26:19.512 INFO:teuthology.orchestra.run.smithi063.stdout:1153 write 0x3b291cb thru 0x3b360dd (0xcf13 bytes) 2024-09-13T09:26:19.518 INFO:teuthology.orchestra.run.smithi063.stdout:1154 write 0x2ed23c2 thru 0x2ed3911 (0x1550 bytes) 2024-09-13T09:26:19.524 INFO:teuthology.orchestra.run.smithi063.stdout:1155 read 0x8d5df19 thru 0x8d5ec88 (0xd70 bytes) 2024-09-13T09:26:19.524 INFO:teuthology.orchestra.run.smithi063.stdout:1157 write 0x6b70070 thru 0x6b7cd46 (0xccd7 bytes) 2024-09-13T09:26:19.539 INFO:teuthology.orchestra.run.smithi063.stdout:1158 writesame 0xd8350b7 thru 0xd837734 data_size 0xd(0x267e bytes) 2024-09-13T09:26:19.539 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.544 INFO:teuthology.orchestra.run.smithi063.stdout:1160 trunc from 0xd837735 to 0xa05228d 2024-09-13T09:26:19.570 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.586 INFO:teuthology.orchestra.run.smithi063.stdout:1162 write 0xccdcf64 thru 0xcce6687 (0x9724 bytes) 2024-09-13T09:26:19.591 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:19.592 INFO:teuthology.orchestra.run.smithi063.stdout:1163 read 0xa5c2cfe thru 0xa5cc75a (0x9a5d bytes) 2024-09-13T09:26:19.593 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.593 INFO:teuthology.orchestra.run.smithi063.stdout:1166 write 0x9095dcb thru 0x90996e8 (0x391e bytes) 2024-09-13T09:26:19.599 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.599 INFO:teuthology.orchestra.run.smithi063.stdout:1169 read 0xbcd3fb2 thru 0xbce2281 (0xe2d0 bytes) 2024-09-13T09:26:19.599 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:19.595+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:19.599 INFO:teuthology.orchestra.run.smithi063.stdout:1170 writesame 0xae0aaea thru 0xae0fc15 data_size 0xa(0x512c bytes) 2024-09-13T09:26:19.608 INFO:teuthology.orchestra.run.smithi063.stdout:1171 trunc from 0xcce6688 to 0xd8282a6 2024-09-13T09:26:19.612 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:19.617 INFO:teuthology.orchestra.run.smithi063.stdout:1172 writesame 0x849c69d thru 0x84a1f38 data_size 0x4(0x589c bytes) 2024-09-13T09:26:19.626 INFO:teuthology.orchestra.run.smithi063.stdout:1173 punch from 0x593d6ef to 0x594819c, (0xaaad bytes) 2024-09-13T09:26:19.626 INFO:teuthology.orchestra.run.smithi063.stdout:1174 punch from 0x198976a to 0x1998f0c, (0xf7a2 bytes) 2024-09-13T09:26:19.632 INFO:teuthology.orchestra.run.smithi063.stdout:1175 trunc from 0xd8282a6 to 0xa07e32e 2024-09-13T09:26:19.635 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:19.640 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:19.640 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:19.650 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:19.653 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.653 INFO:teuthology.orchestra.run.smithi063.stdout:1179 write 0x73f82f2 thru 0x73fe6ed (0x63fc bytes) 2024-09-13T09:26:19.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:19.658 INFO:teuthology.orchestra.run.smithi063.stdout:1181 write 0x66654b6 thru 0x666abba (0x5705 bytes) 2024-09-13T09:26:19.664 INFO:teuthology.orchestra.run.smithi063.stdout:1184 write 0x538c9c thru 0x5465d0 (0xd935 bytes) 2024-09-13T09:26:19.668 INFO:teuthology.orchestra.run.smithi063.stdout:1186 read 0x79f638e thru 0x7a049cf (0xe642 bytes) 2024-09-13T09:26:19.668 INFO:teuthology.orchestra.run.smithi063.stdout:1187 punch from 0x5043123 to 0x5052617, (0xf4f4 bytes) 2024-09-13T09:26:19.668 INFO:teuthology.orchestra.run.smithi063.stdout:1189 read 0xfbbc6d thru 0xfc2b80 (0x6f14 bytes) 2024-09-13T09:26:19.668 INFO:teuthology.orchestra.run.smithi063.stdout:1190 punch from 0x4dd2d3f to 0x4dd3e62, (0x1123 bytes) 2024-09-13T09:26:19.668 INFO:teuthology.orchestra.run.smithi063.stdout:1191 punch from 0x3c7b050 to 0x3c7dcc6, (0x2c76 bytes) 2024-09-13T09:26:19.668 INFO:teuthology.orchestra.run.smithi063.stdout:1192 writesame 0x39e02ba thru 0x39e500b data_size 0x3(0x4d52 bytes) 2024-09-13T09:26:19.677 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:19.678 INFO:teuthology.orchestra.run.smithi063.stdout:1193 flatten 2024-09-13T09:26:19.687 INFO:teuthology.orchestra.run.smithi063.stdout:1194 write 0x7e2381a thru 0x7e26486 (0x2c6d bytes) 2024-09-13T09:26:19.699 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:19.699 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:19.695+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:19.708 INFO:teuthology.orchestra.run.smithi063.stdout:1197 trunc from 0xa07e32e to 0x2b84eee 2024-09-13T09:26:19.722 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:19.729 INFO:teuthology.orchestra.run.smithi063.stdout:1198 read 0x2681c95 thru 0x268ab3e (0x8eaa bytes) 2024-09-13T09:26:19.729 INFO:teuthology.orchestra.run.smithi063.stdout:1200 punch from 0xd39514 to 0xd3a2d4, (0xdc0 bytes) 2024-09-13T09:26:19.729 INFO:teuthology.orchestra.run.smithi063.stdout:1201 read 0x2a98741 thru 0x2aa404c (0xb90c bytes) 2024-09-13T09:26:19.729 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.729 INFO:teuthology.orchestra.run.smithi063.stdout:1203 punch from 0x256ea5c to 0x257a9b1, (0xbf55 bytes) 2024-09-13T09:26:19.730 INFO:teuthology.orchestra.run.smithi063.stdout:1204 punch from 0x290b5d1 to 0x291a231, (0xec60 bytes) 2024-09-13T09:26:19.743 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:19.745 INFO:teuthology.orchestra.run.smithi063.stdout:1206 trunc from 0x2b84eee to 0xc9fc260 2024-09-13T09:26:19.745 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:19.745 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:19.745 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:19.746 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:19.746 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:19.746 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:19.746 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:19.746 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:19.755 INFO:teuthology.orchestra.run.smithi063.stdout:1207 punch from 0x30e64b to 0x3169b6, (0x836b bytes) 2024-09-13T09:26:19.756 INFO:teuthology.orchestra.run.smithi063.stdout:1208 trunc from 0xc9fc260 to 0xc14ae43 2024-09-13T09:26:19.767 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:19.767 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:19.768 INFO:teuthology.orchestra.run.smithi063.stdout:1209 punch from 0x6b8f2ab to 0x6b98253, (0x8fa8 bytes) 2024-09-13T09:26:19.768 INFO:teuthology.orchestra.run.smithi063.stdout:1210 read 0x82f70e2 thru 0x82f8404 (0x1323 bytes) 2024-09-13T09:26:19.768 INFO:teuthology.orchestra.run.smithi063.stdout:1211 writesame 0x5d8fc94 thru 0x5d999da data_size 0x1(0x9d47 bytes) 2024-09-13T09:26:19.778 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:19.784 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:26:19.786 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:26:19.800 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-13T09:26:19.388261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399000740,"num_objects":274,"num_object_clones":70,"num_object_copies":548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":274,"num_whiteouts":64,"num_read":17153,"num_read_kb":45895,"num_write":2951,"num_write_kb":55638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":35,"num_keys_recovered":8,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2431,"ondisk_log_size":2431,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":227068,"kb_used_data":9188,"kb_used_omap":10,"kb_used_meta":217717,"kb_avail":660375812,"statfs":{"total":676457349120,"available":676224831488,"internally_reserved":0,"allocated":9408512,"data_stored":35040828,"data_compressed":1396166,"data_compressed_allocated":1433600,"data_compressed_original":29237164,"omap_allocated":11150,"internal_metadata":222942322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":154283193,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":19,"num_read":6663,"num_read_kb":15716,"num_write":820,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029322"},"pg_stats":[{"pgid":"3.1f","version":"40'39","reported_seq":111,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.699789+0000","last_change":"2024-09-13T09:25:58.217837+0000","last_active":"2024-09-13T09:26:15.699789+0000","last_peered":"2024-09-13T09:26:15.699789+0000","last_clean":"2024-09-13T09:26:15.699789+0000","last_became_active":"2024-09-13T09:25:58.208322+0000","last_became_peered":"2024-09-13T09:25:58.208322+0000","last_unstale":"2024-09-13T09:26:15.699789+0000","last_undegraded":"2024-09-13T09:26:15.699789+0000","last_fullsized":"2024-09-13T09:26:15.699789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:55.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":9655042,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":58,"num_read_kb":205,"num_write":35,"num_write_kb":557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"41'41","reported_seq":116,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.433376+0000","last_change":"2024-09-13T09:25:55.208452+0000","last_active":"2024-09-13T09:26:16.433376+0000","last_peered":"2024-09-13T09:26:16.433376+0000","last_clean":"2024-09-13T09:26:16.433376+0000","last_became_active":"2024-09-13T09:25:53.175729+0000","last_became_peered":"2024-09-13T09:25:53.175729+0000","last_unstale":"2024-09-13T09:26:16.433376+0000","last_undegraded":"2024-09-13T09:26:16.433376+0000","last_fullsized":"2024-09-13T09:26:16.433376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:26.138376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":6230395,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":64,"num_read_kb":306,"num_write":47,"num_write_kb":633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"41'33","reported_seq":157,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.379257+0000","last_change":"2024-09-13T09:25:55.200905+0000","last_active":"2024-09-13T09:26:16.379257+0000","last_peered":"2024-09-13T09:26:16.379257+0000","last_clean":"2024-09-13T09:26:16.379257+0000","last_became_active":"2024-09-13T09:25:53.182525+0000","last_became_peered":"2024-09-13T09:25:53.182525+0000","last_unstale":"2024-09-13T09:26:16.379257+0000","last_undegraded":"2024-09-13T09:26:16.379257+0000","last_fullsized":"2024-09-13T09:26:16.379257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:56.196277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":17233013,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":121,"num_read_kb":270,"num_write":30,"num_write_kb":259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"41'181","reported_seq":356,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.439145+0000","last_change":"2024-09-13T09:25:55.200942+0000","last_active":"2024-09-13T09:26:16.439145+0000","last_peered":"2024-09-13T09:26:16.439145+0000","last_clean":"2024-09-13T09:26:16.439145+0000","last_became_active":"2024-09-13T09:25:53.182441+0000","last_became_peered":"2024-09-13T09:25:53.182441+0000","last_unstale":"2024-09-13T09:26:16.439145+0000","last_undegraded":"2024-09-13T09:26:16.439145+0000","last_fullsized":"2024-09-13T09:26:16.439145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:00.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":15318337,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":862,"num_read_kb":1159,"num_write":286,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"41'18","reported_seq":567,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.555836+0000","last_change":"2024-09-13T09:25:58.206915+0000","last_active":"2024-09-13T09:26:16.555836+0000","last_peered":"2024-09-13T09:26:16.555836+0000","last_clean":"2024-09-13T09:26:16.555836+0000","last_became_active":"2024-09-13T09:25:58.206640+0000","last_became_peered":"2024-09-13T09:25:58.206640+0000","last_unstale":"2024-09-13T09:26:16.555836+0000","last_undegraded":"2024-09-13T09:26:16.555836+0000","last_fullsized":"2024-09-13T09:26:16.555836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:24.364502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1640961,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":552,"num_read_kb":1374,"num_write":20,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"41'82","reported_seq":1420,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.434501+0000","last_change":"2024-09-13T09:25:55.187594+0000","last_active":"2024-09-13T09:26:16.434501+0000","last_peered":"2024-09-13T09:26:16.434501+0000","last_clean":"2024-09-13T09:26:16.434501+0000","last_became_active":"2024-09-13T09:25:53.167913+0000","last_became_peered":"2024-09-13T09:25:53.167913+0000","last_unstale":"2024-09-13T09:26:16.434501+0000","last_undegraded":"2024-09-13T09:26:16.434501+0000","last_fullsized":"2024-09-13T09:26:16.434501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:18:31.544056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":29457717,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1854,"num_read_kb":5869,"num_write":81,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"41'61","reported_seq":189,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.565826+0000","last_change":"2024-09-13T09:25:55.184140+0000","last_active":"2024-09-13T09:26:16.565826+0000","last_peered":"2024-09-13T09:26:16.565826+0000","last_clean":"2024-09-13T09:26:16.565826+0000","last_became_active":"2024-09-13T09:25:53.166440+0000","last_became_peered":"2024-09-13T09:25:53.166440+0000","last_unstale":"2024-09-13T09:26:16.565826+0000","last_undegraded":"2024-09-13T09:26:16.565826+0000","last_fullsized":"2024-09-13T09:26:16.565826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:23.151437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":15777693,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":203,"num_read_kb":283,"num_write":53,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"41'43","reported_seq":287,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.437437+0000","last_change":"2024-09-13T09:25:58.216595+0000","last_active":"2024-09-13T09:26:16.437437+0000","last_peered":"2024-09-13T09:26:16.437437+0000","last_clean":"2024-09-13T09:26:16.437437+0000","last_became_active":"2024-09-13T09:25:58.202391+0000","last_became_peered":"2024-09-13T09:25:58.202391+0000","last_unstale":"2024-09-13T09:26:16.437437+0000","last_undegraded":"2024-09-13T09:26:16.437437+0000","last_fullsized":"2024-09-13T09:26:16.437437+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:56.591120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":717784,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":582,"num_read_kb":537,"num_write":34,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":8,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"41'162","reported_seq":197,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.442212+0000","last_change":"2024-09-13T09:25:58.211250+0000","last_active":"2024-09-13T09:26:16.442212+0000","last_peered":"2024-09-13T09:26:16.442212+0000","last_clean":"2024-09-13T09:26:16.442212+0000","last_became_active":"2024-09-13T09:25:58.210896+0000","last_became_peered":"2024-09-13T09:25:58.210896+0000","last_unstale":"2024-09-13T09:26:16.442212+0000","last_undegraded":"2024-09-13T09:26:16.442212+0000","last_fullsized":"2024-09-13T09:26:16.442212+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:29:33.397561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":704,"num_read_kb":548,"num_write":248,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"41'96","reported_seq":300,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.407902+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:26:16.407902+0000","last_peered":"2024-09-13T09:26:16.407902+0000","last_clean":"2024-09-13T09:26:16.407902+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:26:16.407902+0000","last_undegraded":"2024-09-13T09:26:16.407902+0000","last_fullsized":"2024-09-13T09:26:16.407902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":23429155,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":494,"num_read_kb":6049,"num_write":129,"num_write_kb":1507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"41'23","reported_seq":68,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.387762+0000","last_change":"2024-09-13T09:25:55.218654+0000","last_active":"2024-09-13T09:26:16.387762+0000","last_peered":"2024-09-13T09:26:16.387762+0000","last_clean":"2024-09-13T09:26:16.387762+0000","last_became_active":"2024-09-13T09:25:53.176649+0000","last_became_peered":"2024-09-13T09:25:53.176649+0000","last_unstale":"2024-09-13T09:26:16.387762+0000","last_undegraded":"2024-09-13T09:26:16.387762+0000","last_fullsized":"2024-09-13T09:26:16.387762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:24.563411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":4616266,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":15,"num_write":22,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"41'37","reported_seq":279,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.386653+0000","last_change":"2024-09-13T09:25:55.188342+0000","last_active":"2024-09-13T09:26:16.386653+0000","last_peered":"2024-09-13T09:26:16.386653+0000","last_clean":"2024-09-13T09:26:16.386653+0000","last_became_active":"2024-09-13T09:25:53.178133+0000","last_became_peered":"2024-09-13T09:25:53.178133+0000","last_unstale":"2024-09-13T09:26:16.386653+0000","last_undegraded":"2024-09-13T09:26:16.386653+0000","last_fullsized":"2024-09-13T09:26:16.386653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:01.188801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":19419996,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":232,"num_read_kb":3836,"num_write":30,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.362222+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:26:15.362222+0000","last_peered":"2024-09-13T09:26:15.362222+0000","last_clean":"2024-09-13T09:26:15.362222+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:26:15.362222+0000","last_undegraded":"2024-09-13T09:26:15.362222+0000","last_fullsized":"2024-09-13T09:26:15.362222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"38'34","reported_seq":143,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.849146+0000","last_change":"2024-09-13T09:25:55.201311+0000","last_active":"2024-09-13T09:26:15.849146+0000","last_peered":"2024-09-13T09:26:15.849146+0000","last_clean":"2024-09-13T09:26:15.849146+0000","last_became_active":"2024-09-13T09:25:53.175817+0000","last_became_peered":"2024-09-13T09:25:53.175817+0000","last_unstale":"2024-09-13T09:26:15.849146+0000","last_undegraded":"2024-09-13T09:26:15.849146+0000","last_fullsized":"2024-09-13T09:26:15.849146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:24:01.314273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":18476436,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":104,"num_read_kb":210,"num_write":30,"num_write_kb":287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.362174+0000","last_change":"2024-09-13T09:25:58.203812+0000","last_active":"2024-09-13T09:26:15.362174+0000","last_peered":"2024-09-13T09:26:15.362174+0000","last_clean":"2024-09-13T09:26:15.362174+0000","last_became_active":"2024-09-13T09:25:58.203492+0000","last_became_peered":"2024-09-13T09:25:58.203492+0000","last_unstale":"2024-09-13T09:26:15.362174+0000","last_undegraded":"2024-09-13T09:26:15.362174+0000","last_fullsized":"2024-09-13T09:26:15.362174+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:12.981969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"41'176","reported_seq":833,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.408810+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:26:16.408810+0000","last_peered":"2024-09-13T09:26:16.408810+0000","last_clean":"2024-09-13T09:26:16.408810+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:26:16.408810+0000","last_undegraded":"2024-09-13T09:26:16.408810+0000","last_fullsized":"2024-09-13T09:26:16.408810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":262179,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2207,"num_read_kb":1938,"num_write":224,"num_write_kb":12599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.368933+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:26:15.368933+0000","last_peered":"2024-09-13T09:26:15.368933+0000","last_clean":"2024-09-13T09:26:15.368933+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:26:15.368933+0000","last_undegraded":"2024-09-13T09:26:15.368933+0000","last_fullsized":"2024-09-13T09:26:15.368933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"41'41","reported_seq":151,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.367473+0000","last_change":"2024-09-13T09:25:55.208924+0000","last_active":"2024-09-13T09:26:16.367473+0000","last_peered":"2024-09-13T09:26:16.367473+0000","last_clean":"2024-09-13T09:26:16.367473+0000","last_became_active":"2024-09-13T09:25:53.171139+0000","last_became_peered":"2024-09-13T09:25:53.171139+0000","last_unstale":"2024-09-13T09:26:16.367473+0000","last_undegraded":"2024-09-13T09:26:16.367473+0000","last_fullsized":"2024-09-13T09:26:16.367473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:51.468156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":20242718,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":94,"num_read_kb":187,"num_write":45,"num_write_kb":366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.358356+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:26:15.358356+0000","last_peered":"2024-09-13T09:26:15.358356+0000","last_clean":"2024-09-13T09:26:15.358356+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:26:15.358356+0000","last_undegraded":"2024-09-13T09:26:15.358356+0000","last_fullsized":"2024-09-13T09:26:15.358356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"41'71","reported_seq":514,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.436321+0000","last_change":"2024-09-13T09:25:58.209367+0000","last_active":"2024-09-13T09:26:16.436321+0000","last_peered":"2024-09-13T09:26:16.436321+0000","last_clean":"2024-09-13T09:26:16.436321+0000","last_became_active":"2024-09-13T09:25:58.209013+0000","last_became_peered":"2024-09-13T09:25:58.209013+0000","last_unstale":"2024-09-13T09:26:16.436321+0000","last_undegraded":"2024-09-13T09:26:16.436321+0000","last_fullsized":"2024-09-13T09:26:16.436321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:01.150567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":832094,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1244,"num_read_kb":1090,"num_write":64,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"40'57","reported_seq":416,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.319717+0000","last_change":"2024-09-13T09:25:58.209033+0000","last_active":"2024-09-13T09:26:16.319717+0000","last_peered":"2024-09-13T09:26:16.319717+0000","last_clean":"2024-09-13T09:26:16.319717+0000","last_became_active":"2024-09-13T09:25:58.208751+0000","last_became_peered":"2024-09-13T09:25:58.208751+0000","last_unstale":"2024-09-13T09:26:16.319717+0000","last_undegraded":"2024-09-13T09:26:16.319717+0000","last_fullsized":"2024-09-13T09:26:16.319717+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.999004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1019,"num_read_kb":862,"num_write":52,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":181,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.430577+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:26:15.430577+0000","last_peered":"2024-09-13T09:26:15.430577+0000","last_clean":"2024-09-13T09:26:15.430577+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:26:15.430577+0000","last_undegraded":"2024-09-13T09:26:15.430577+0000","last_fullsized":"2024-09-13T09:26:15.430577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":39,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.472927+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:26:15.472927+0000","last_peered":"2024-09-13T09:26:15.472927+0000","last_clean":"2024-09-13T09:26:15.472927+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:26:15.472927+0000","last_undegraded":"2024-09-13T09:26:15.472927+0000","last_fullsized":"2024-09-13T09:26:15.472927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"41'89","reported_seq":454,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.408411+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:26:16.408411+0000","last_peered":"2024-09-13T09:26:16.408411+0000","last_clean":"2024-09-13T09:26:16.408411+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:26:16.408411+0000","last_undegraded":"2024-09-13T09:26:16.408411+0000","last_fullsized":"2024-09-13T09:26:16.408411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":19737358,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":875,"num_read_kb":801,"num_write":80,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.430709+0000","last_change":"2024-09-13T09:25:58.215642+0000","last_active":"2024-09-13T09:26:15.430709+0000","last_peered":"2024-09-13T09:26:15.430709+0000","last_clean":"2024-09-13T09:26:15.430709+0000","last_became_active":"2024-09-13T09:25:58.211664+0000","last_became_peered":"2024-09-13T09:25:58.211664+0000","last_unstale":"2024-09-13T09:26:15.430709+0000","last_undegraded":"2024-09-13T09:26:15.430709+0000","last_fullsized":"2024-09-13T09:26:15.430709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:40.364839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"41'59","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.557702+0000","last_change":"2024-09-13T09:25:58.212397+0000","last_active":"2024-09-13T09:26:16.557702+0000","last_peered":"2024-09-13T09:26:16.557702+0000","last_clean":"2024-09-13T09:26:16.557702+0000","last_became_active":"2024-09-13T09:25:58.212107+0000","last_became_peered":"2024-09-13T09:25:58.212107+0000","last_unstale":"2024-09-13T09:26:16.557702+0000","last_undegraded":"2024-09-13T09:26:16.557702+0000","last_fullsized":"2024-09-13T09:26:16.557702+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:37.963201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":32254173,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":109,"num_read_kb":2506,"num_write":63,"num_write_kb":1101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":46,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.373904+0000","last_change":"2024-09-13T09:25:58.210139+0000","last_active":"2024-09-13T09:26:15.373904+0000","last_peered":"2024-09-13T09:26:15.373904+0000","last_clean":"2024-09-13T09:26:15.373904+0000","last_became_active":"2024-09-13T09:25:58.209840+0000","last_became_peered":"2024-09-13T09:25:58.209840+0000","last_unstale":"2024-09-13T09:26:15.373904+0000","last_undegraded":"2024-09-13T09:26:15.373904+0000","last_fullsized":"2024-09-13T09:26:15.373904+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.660609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"40'26","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.318354+0000","last_change":"2024-09-13T09:25:55.209051+0000","last_active":"2024-09-13T09:26:16.318354+0000","last_peered":"2024-09-13T09:26:16.318354+0000","last_clean":"2024-09-13T09:26:16.318354+0000","last_became_active":"2024-09-13T09:25:53.177322+0000","last_became_peered":"2024-09-13T09:25:53.177322+0000","last_unstale":"2024-09-13T09:26:16.318354+0000","last_undegraded":"2024-09-13T09:26:16.318354+0000","last_fullsized":"2024-09-13T09:26:16.318354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:04.763964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":3953922,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":71,"num_read_kb":107,"num_write":23,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.430637+0000","last_change":"2024-09-13T09:25:58.204165+0000","last_active":"2024-09-13T09:26:15.430637+0000","last_peered":"2024-09-13T09:26:15.430637+0000","last_clean":"2024-09-13T09:26:15.430637+0000","last_became_active":"2024-09-13T09:25:58.203817+0000","last_became_peered":"2024-09-13T09:25:58.203817+0000","last_unstale":"2024-09-13T09:26:15.430637+0000","last_undegraded":"2024-09-13T09:26:15.430637+0000","last_fullsized":"2024-09-13T09:26:15.430637+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.258811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"41'99","reported_seq":181,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.431577+0000","last_change":"2024-09-13T09:25:55.208255+0000","last_active":"2024-09-13T09:26:16.431577+0000","last_peered":"2024-09-13T09:26:16.431577+0000","last_clean":"2024-09-13T09:26:16.431577+0000","last_became_active":"2024-09-13T09:25:53.167023+0000","last_became_peered":"2024-09-13T09:25:53.167023+0000","last_unstale":"2024-09-13T09:26:16.431577+0000","last_undegraded":"2024-09-13T09:26:16.431577+0000","last_fullsized":"2024-09-13T09:26:16.431577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:15.699046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":3450451,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":382,"num_read_kb":461,"num_write":137,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"41'53","reported_seq":130,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.444907+0000","last_change":"2024-09-13T09:25:55.187602+0000","last_active":"2024-09-13T09:26:16.444907+0000","last_peered":"2024-09-13T09:26:16.444907+0000","last_clean":"2024-09-13T09:26:16.444907+0000","last_became_active":"2024-09-13T09:25:53.646396+0000","last_became_peered":"2024-09-13T09:25:53.646396+0000","last_unstale":"2024-09-13T09:26:16.444907+0000","last_undegraded":"2024-09-13T09:26:16.444907+0000","last_fullsized":"2024-09-13T09:26:16.444907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:21.102946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":18851621,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":67,"num_read_kb":266,"num_write":50,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"40'22","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.562483+0000","last_change":"2024-09-13T09:25:55.752827+0000","last_active":"2024-09-13T09:26:15.562483+0000","last_peered":"2024-09-13T09:26:15.562483+0000","last_clean":"2024-09-13T09:26:15.562483+0000","last_became_active":"2024-09-13T09:25:53.180652+0000","last_became_peered":"2024-09-13T09:25:53.180652+0000","last_unstale":"2024-09-13T09:26:15.562483+0000","last_undegraded":"2024-09-13T09:26:15.562483+0000","last_fullsized":"2024-09-13T09:26:15.562483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:31.395047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":4390912,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":21,"num_read_kb":153,"num_write":21,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"41'64","reported_seq":247,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.434032+0000","last_change":"2024-09-13T09:25:58.204397+0000","last_active":"2024-09-13T09:26:16.434032+0000","last_peered":"2024-09-13T09:26:16.434032+0000","last_clean":"2024-09-13T09:26:16.434032+0000","last_became_active":"2024-09-13T09:25:58.204077+0000","last_became_peered":"2024-09-13T09:25:58.204077+0000","last_unstale":"2024-09-13T09:26:16.434032+0000","last_undegraded":"2024-09-13T09:26:16.434032+0000","last_fullsized":"2024-09-13T09:26:16.434032+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:52:46.564832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":41276601,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":187,"num_read_kb":6154,"num_write":78,"num_write_kb":13736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"41'241","reported_seq":345,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.379070+0000","last_change":"2024-09-13T09:25:55.199660+0000","last_active":"2024-09-13T09:26:16.379070+0000","last_peered":"2024-09-13T09:26:16.379070+0000","last_clean":"2024-09-13T09:26:16.379070+0000","last_became_active":"2024-09-13T09:25:53.179342+0000","last_became_peered":"2024-09-13T09:25:53.179342+0000","last_unstale":"2024-09-13T09:26:16.379070+0000","last_undegraded":"2024-09-13T09:26:16.379070+0000","last_fullsized":"2024-09-13T09:26:16.379070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:50.835430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":16646476,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1101,"num_read_kb":1225,"num_write":386,"num_write_kb":856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"41'52","reported_seq":112,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.387760+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:26:16.387760+0000","last_peered":"2024-09-13T09:26:16.387760+0000","last_clean":"2024-09-13T09:26:16.387760+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:26:16.387760+0000","last_undegraded":"2024-09-13T09:26:16.387760+0000","last_fullsized":"2024-09-13T09:26:16.387760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":6284030,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":55,"num_read_kb":2188,"num_write":63,"num_write_kb":12809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"41'41","reported_seq":104,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.440527+0000","last_change":"2024-09-13T09:25:55.220725+0000","last_active":"2024-09-13T09:26:16.440527+0000","last_peered":"2024-09-13T09:26:16.440527+0000","last_clean":"2024-09-13T09:26:16.440527+0000","last_became_active":"2024-09-13T09:25:53.178164+0000","last_became_peered":"2024-09-13T09:25:53.178164+0000","last_unstale":"2024-09-13T09:26:16.440527+0000","last_undegraded":"2024-09-13T09:26:16.440527+0000","last_fullsized":"2024-09-13T09:26:16.440527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:52.167054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":13217289,"num_objects":13,"num_object_clones":6,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":56,"num_read_kb":603,"num_write":32,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"41'43","reported_seq":116,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.434821+0000","last_change":"2024-09-13T09:25:55.219599+0000","last_active":"2024-09-13T09:26:16.434821+0000","last_peered":"2024-09-13T09:26:16.434821+0000","last_clean":"2024-09-13T09:26:16.434821+0000","last_became_active":"2024-09-13T09:25:53.177574+0000","last_became_peered":"2024-09-13T09:25:53.177574+0000","last_unstale":"2024-09-13T09:26:16.434821+0000","last_undegraded":"2024-09-13T09:26:16.434821+0000","last_fullsized":"2024-09-13T09:26:16.434821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:35.248811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":5411450,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":81,"num_read_kb":145,"num_write":36,"num_write_kb":403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"41'92","reported_seq":161,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.558850+0000","last_change":"2024-09-13T09:25:58.206682+0000","last_active":"2024-09-13T09:26:16.558850+0000","last_peered":"2024-09-13T09:26:16.558850+0000","last_clean":"2024-09-13T09:26:16.558850+0000","last_became_active":"2024-09-13T09:25:58.206323+0000","last_became_peered":"2024-09-13T09:25:58.206323+0000","last_unstale":"2024-09-13T09:26:16.558850+0000","last_undegraded":"2024-09-13T09:26:16.558850+0000","last_fullsized":"2024-09-13T09:26:16.558850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:46.537802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":10854705,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":415,"num_read_kb":820,"num_write":149,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"40'65","reported_seq":1130,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.263331+0000","last_change":"2024-09-13T09:25:55.219871+0000","last_active":"2024-09-13T09:26:16.263331+0000","last_peered":"2024-09-13T09:26:16.263331+0000","last_clean":"2024-09-13T09:26:16.263331+0000","last_became_active":"2024-09-13T09:25:53.179182+0000","last_became_peered":"2024-09-13T09:25:53.179182+0000","last_unstale":"2024-09-13T09:26:16.263331+0000","last_undegraded":"2024-09-13T09:26:16.263331+0000","last_fullsized":"2024-09-13T09:26:16.263331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:45.198757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":15431740,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1148,"num_read_kb":3741,"num_write":69,"num_write_kb":2040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"41'72","reported_seq":383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.471989+0000","last_change":"2024-09-13T09:25:58.209400+0000","last_active":"2024-09-13T09:26:16.471989+0000","last_peered":"2024-09-13T09:26:16.471989+0000","last_clean":"2024-09-13T09:26:16.471989+0000","last_became_active":"2024-09-13T09:25:58.209004+0000","last_became_peered":"2024-09-13T09:25:58.209004+0000","last_unstale":"2024-09-13T09:26:16.471989+0000","last_undegraded":"2024-09-13T09:26:16.471989+0000","last_fullsized":"2024-09-13T09:26:16.471989+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:25.744656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":748236,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":950,"num_read_kb":854,"num_write":84,"num_write_kb":205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"41'98","reported_seq":578,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.443522+0000","last_change":"2024-09-13T09:25:55.219148+0000","last_active":"2024-09-13T09:26:16.443522+0000","last_peered":"2024-09-13T09:26:16.443522+0000","last_clean":"2024-09-13T09:26:16.443522+0000","last_became_active":"2024-09-13T09:25:53.177434+0000","last_became_peered":"2024-09-13T09:25:53.177434+0000","last_unstale":"2024-09-13T09:26:16.443522+0000","last_undegraded":"2024-09-13T09:26:16.443522+0000","last_fullsized":"2024-09-13T09:26:16.443522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:13:59.794157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":22591500,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1094,"num_read_kb":1033,"num_write":89,"num_write_kb":632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":398410353,"num_objects":270,"num_object_clones":70,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":64,"num_read":17037,"num_read_kb":45795,"num_write":2790,"num_write_kb":53220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":16,"num_keys_recovered":8,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6057984,"data_stored":32062046,"data_compressed":1308692,"data_compressed_allocated":1261568,"data_compressed_original":27639724,"omap_allocated":0,"internal_metadata":0},"log_size":2311,"ondisk_log_size":2311,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2105344,"data_stored":9614640,"data_compressed":385940,"data_compressed_allocated":372736,"data_compressed_original":8052736,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":798720,"data_stored":1183626,"data_compressed":34431,"data_compressed_allocated":45056,"data_compressed_original":729088,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1359872,"data_stored":5106231,"data_compressed":205384,"data_compressed_allocated":200704,"data_compressed_original":4308950,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2174976,"data_stored":10446044,"data_compressed":423520,"data_compressed_allocated":393216,"data_compressed_original":8970240,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1351680,"data_stored":6019298,"data_compressed":250548,"data_compressed_allocated":253952,"data_compressed_original":5304320,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":921600,"data_stored":1653634,"data_compressed":62693,"data_compressed_allocated":118784,"data_compressed_original":1200128,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":0,"up_from":0,"seq":64424509449,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":696320,"data_stored":1017355,"data_compressed":33650,"data_compressed_allocated":49152,"data_compressed_original":671702,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":628372,"data_compressed":18956,"data_compressed_allocated":32768,"data_compressed_original":401408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":838481,"data_compressed":33650,"data_compressed_allocated":49152,"data_compressed_original":671702,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":913408,"data_stored":5835196,"data_compressed":250548,"data_compressed_allocated":253952,"data_compressed_original":5304320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1728512,"data_stored":10260987,"data_compressed":423520,"data_compressed_allocated":393216,"data_compressed_original":8970240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":921600,"data_stored":4922102,"data_compressed":205384,"data_compressed_allocated":200704,"data_compressed_original":4308950,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":998561,"data_compressed":34431,"data_compressed_allocated":45056,"data_compressed_original":729088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1515520,"data_stored":8578331,"data_compressed":342203,"data_compressed_allocated":286720,"data_compressed_original":7254016,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:26:19.803 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:26:19.804 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:19.797+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:19.804 INFO:teuthology.orchestra.run.smithi063.stdout:1212 punch from 0xbe561c7 to 0xbe5809c, (0x1ed5 bytes) 2024-09-13T09:26:19.804 INFO:teuthology.orchestra.run.smithi063.stdout:1214 punch from 0x2d5d4b6 to 0x2d63711, (0x625b bytes) 2024-09-13T09:26:19.804 INFO:teuthology.orchestra.run.smithi063.stdout:1215 writesame 0x5e25c1a thru 0x5e3002c data_size 0x27(0xa413 bytes) 2024-09-13T09:26:19.809 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.809 INFO:teuthology.orchestra.run.smithi063.stdout:1217 read 0x5c36a00 thru 0x5c3bb13 (0x5114 bytes) 2024-09-13T09:26:19.810 INFO:teuthology.orchestra.run.smithi063.stdout:1218 read 0xad301f9 thru 0xad3712f (0x6f37 bytes) 2024-09-13T09:26:19.810 INFO:teuthology.orchestra.run.smithi063.stdout:1219 write 0xbf15ea4 thru 0xbf214d9 (0xb636 bytes) 2024-09-13T09:26:19.821 INFO:teuthology.orchestra.run.smithi063.stdout:1220 trunc from 0xc14ae43 to 0x91a9e6c 2024-09-13T09:26:19.862 INFO:teuthology.orchestra.run.smithi063.stdout:1221 writesame 0xdb50870 thru 0xdb52212 data_size 0x1(0x19a3 bytes) 2024-09-13T09:26:19.862 INFO:teuthology.orchestra.run.smithi063.stdout:1222 read 0x9e2b5cc thru 0x9e36383 (0xadb8 bytes) 2024-09-13T09:26:19.863 INFO:teuthology.orchestra.run.smithi063.stdout:1223 read 0x291c086 thru 0x2926a14 (0xa98f bytes) 2024-09-13T09:26:19.863 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.863 INFO:teuthology.orchestra.run.smithi063.stdout:1225 read 0x380f461 thru 0x380fd64 (0x904 bytes) 2024-09-13T09:26:19.863 INFO:teuthology.orchestra.run.smithi063.stdout:1226 read 0xd9df196 thru 0xd9ebbb9 (0xca24 bytes) 2024-09-13T09:26:19.863 INFO:teuthology.orchestra.run.smithi063.stdout:1227 write 0x3c106c9 thru 0x3c12b9b (0x24d3 bytes) 2024-09-13T09:26:19.885 INFO:teuthology.orchestra.run.smithi063.stdout:1228 trunc from 0xdb52213 to 0x3654c1e 2024-09-13T09:26:19.901 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:19.895+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:19.905 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:19.908 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.908 INFO:teuthology.orchestra.run.smithi063.stdout:1230 read 0xaeb216 thru 0xaf53a9 (0xa194 bytes) 2024-09-13T09:26:19.918 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:19.925 INFO:teuthology.orchestra.run.smithi063.stdout:1234 writesame 0x931b054 thru 0x93212e7 data_size 0x12(0x6294 bytes) 2024-09-13T09:26:19.925 INFO:teuthology.orchestra.run.smithi063.stdout:1235 writesame 0xd42dcd thru 0xd478e6 data_size 0x1(0x4b1a bytes) 2024-09-13T09:26:19.943 INFO:teuthology.orchestra.run.smithi063.stdout:1236 trunc from 0x93212e8 to 0x951bfa9 2024-09-13T09:26:19.955 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:19.965 INFO:teuthology.orchestra.run.smithi063.stdout:1240 trunc from 0x951bfa9 to 0xe716900 2024-09-13T09:26:19.978 INFO:teuthology.orchestra.run.smithi063.stdout:1243 read 0x9e04e2c thru 0x9e10d52 (0xbf27 bytes) 2024-09-13T09:26:19.979 INFO:teuthology.orchestra.run.smithi063.stdout:1246 write 0x60b0f80 thru 0x60bbe3f (0xaec0 bytes) 2024-09-13T09:26:19.984 INFO:teuthology.orchestra.run.smithi063.stdout:1247 writesame 0x29c6afd thru 0x29d50c1 data_size 0x7(0xe5c5 bytes) 2024-09-13T09:26:20.002 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:20.001+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:20.013 INFO:teuthology.orchestra.run.smithi063.stdout:1248 trunc from 0xe716900 to 0x32227ed 2024-09-13T09:26:20.033 INFO:teuthology.orchestra.run.smithi063.stdout:1249 read 0x2ca91f8 thru 0x2cb4573 (0xb37c bytes) 2024-09-13T09:26:20.033 INFO:teuthology.orchestra.run.smithi063.stdout:1250 read 0xc263d0 thru 0xc2b0db (0x4d0c bytes) 2024-09-13T09:26:20.033 INFO:teuthology.orchestra.run.smithi063.stdout:1254 writesame 0xa22081 thru 0xa2e731 data_size 0x5(0xc6b1 bytes) 2024-09-13T09:26:20.041 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.041 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.044 INFO:teuthology.orchestra.run.smithi063.stdout:1255 read 0x1f544ad thru 0x1f5a575 (0x60c9 bytes) 2024-09-13T09:26:20.044 INFO:teuthology.orchestra.run.smithi063.stdout:1256 punch from 0x75af91 to 0x75e3dc, (0x344b bytes) 2024-09-13T09:26:20.044 INFO:teuthology.orchestra.run.smithi063.stdout:1257 read 0x1abb3fd thru 0x1ac6540 (0xb144 bytes) 2024-09-13T09:26:20.044 INFO:teuthology.orchestra.run.smithi063.stdout:1258 read 0xd545be thru 0xd58514 (0x3f57 bytes) 2024-09-13T09:26:20.045 INFO:teuthology.orchestra.run.smithi063.stdout:1259 read 0x1294668 thru 0x1298b29 (0x44c2 bytes) 2024-09-13T09:26:20.047 INFO:teuthology.orchestra.run.smithi063.stdout:1261 trunc from 0x32227ed to 0x20a1086 2024-09-13T09:26:20.053 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:1262 writesame 0x1e109cc thru 0x1e1d44a data_size 0x1(0xca7f bytes) 2024-09-13T09:26:20.114 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:20.111+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:20.143 INFO:teuthology.orchestra.run.smithi063.stdout:1265 write 0xeb7b33f thru 0xeb828f7 (0x75b9 bytes) 2024-09-13T09:26:20.156 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:26:20.157 INFO:teuthology.orchestra.run.smithi063.stdout:1267 trunc from 0xeb828f8 to 0xbe7308c 2024-09-13T09:26:20.159 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:26:20.174 INFO:teuthology.orchestra.run.smithi063.stdout:1270 read 0xbc40d5 thru 0xbcc1bd (0x80e9 bytes) 2024-09-13T09:26:20.175 INFO:teuthology.orchestra.run.smithi063.stdout:1271 read 0x90e73a0 thru 0x90eea13 (0x7674 bytes) 2024-09-13T09:26:20.175 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-13T09:26:19.388261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399000740,"num_objects":274,"num_object_clones":70,"num_object_copies":548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":274,"num_whiteouts":64,"num_read":17153,"num_read_kb":45895,"num_write":2951,"num_write_kb":55638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":35,"num_keys_recovered":8,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2431,"ondisk_log_size":2431,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":227068,"kb_used_data":9188,"kb_used_omap":10,"kb_used_meta":217717,"kb_avail":660375812,"statfs":{"total":676457349120,"available":676224831488,"internally_reserved":0,"allocated":9408512,"data_stored":35040828,"data_compressed":1396166,"data_compressed_allocated":1433600,"data_compressed_original":29237164,"omap_allocated":11150,"internal_metadata":222942322},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":154283193,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":19,"num_read":6663,"num_read_kb":15716,"num_write":820,"num_write_kb":5392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029322"},"pg_stats":[{"pgid":"3.1f","version":"40'39","reported_seq":111,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.699789+0000","last_change":"2024-09-13T09:25:58.217837+0000","last_active":"2024-09-13T09:26:15.699789+0000","last_peered":"2024-09-13T09:26:15.699789+0000","last_clean":"2024-09-13T09:26:15.699789+0000","last_became_active":"2024-09-13T09:25:58.208322+0000","last_became_peered":"2024-09-13T09:25:58.208322+0000","last_unstale":"2024-09-13T09:26:15.699789+0000","last_undegraded":"2024-09-13T09:26:15.699789+0000","last_fullsized":"2024-09-13T09:26:15.699789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:55.361158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":9655042,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":58,"num_read_kb":205,"num_write":35,"num_write_kb":557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":16,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"41'41","reported_seq":116,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.433376+0000","last_change":"2024-09-13T09:25:55.208452+0000","last_active":"2024-09-13T09:26:16.433376+0000","last_peered":"2024-09-13T09:26:16.433376+0000","last_clean":"2024-09-13T09:26:16.433376+0000","last_became_active":"2024-09-13T09:25:53.175729+0000","last_became_peered":"2024-09-13T09:25:53.175729+0000","last_unstale":"2024-09-13T09:26:16.433376+0000","last_undegraded":"2024-09-13T09:26:16.433376+0000","last_fullsized":"2024-09-13T09:26:16.433376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:06:26.138376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":6230395,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":64,"num_read_kb":306,"num_write":47,"num_write_kb":633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"41'33","reported_seq":157,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.379257+0000","last_change":"2024-09-13T09:25:55.200905+0000","last_active":"2024-09-13T09:26:16.379257+0000","last_peered":"2024-09-13T09:26:16.379257+0000","last_clean":"2024-09-13T09:26:16.379257+0000","last_became_active":"2024-09-13T09:25:53.182525+0000","last_became_peered":"2024-09-13T09:25:53.182525+0000","last_unstale":"2024-09-13T09:26:16.379257+0000","last_undegraded":"2024-09-13T09:26:16.379257+0000","last_fullsized":"2024-09-13T09:26:16.379257+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:56.196277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":17233013,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":121,"num_read_kb":270,"num_write":30,"num_write_kb":259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"41'181","reported_seq":356,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.439145+0000","last_change":"2024-09-13T09:25:55.200942+0000","last_active":"2024-09-13T09:26:16.439145+0000","last_peered":"2024-09-13T09:26:16.439145+0000","last_clean":"2024-09-13T09:26:16.439145+0000","last_became_active":"2024-09-13T09:25:53.182441+0000","last_became_peered":"2024-09-13T09:25:53.182441+0000","last_unstale":"2024-09-13T09:26:16.439145+0000","last_undegraded":"2024-09-13T09:26:16.439145+0000","last_fullsized":"2024-09-13T09:26:16.439145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:00.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":15318337,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":862,"num_read_kb":1159,"num_write":286,"num_write_kb":784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"41'18","reported_seq":567,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.555836+0000","last_change":"2024-09-13T09:25:58.206915+0000","last_active":"2024-09-13T09:26:16.555836+0000","last_peered":"2024-09-13T09:26:16.555836+0000","last_clean":"2024-09-13T09:26:16.555836+0000","last_became_active":"2024-09-13T09:25:58.206640+0000","last_became_peered":"2024-09-13T09:25:58.206640+0000","last_unstale":"2024-09-13T09:26:16.555836+0000","last_undegraded":"2024-09-13T09:26:16.555836+0000","last_fullsized":"2024-09-13T09:26:16.555836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:24.364502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1640961,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":552,"num_read_kb":1374,"num_write":20,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"41'82","reported_seq":1420,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.434501+0000","last_change":"2024-09-13T09:25:55.187594+0000","last_active":"2024-09-13T09:26:16.434501+0000","last_peered":"2024-09-13T09:26:16.434501+0000","last_clean":"2024-09-13T09:26:16.434501+0000","last_became_active":"2024-09-13T09:25:53.167913+0000","last_became_peered":"2024-09-13T09:25:53.167913+0000","last_unstale":"2024-09-13T09:26:16.434501+0000","last_undegraded":"2024-09-13T09:26:16.434501+0000","last_fullsized":"2024-09-13T09:26:16.434501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:18:31.544056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":29457717,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1854,"num_read_kb":5869,"num_write":81,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"41'61","reported_seq":189,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.565826+0000","last_change":"2024-09-13T09:25:55.184140+0000","last_active":"2024-09-13T09:26:16.565826+0000","last_peered":"2024-09-13T09:26:16.565826+0000","last_clean":"2024-09-13T09:26:16.565826+0000","last_became_active":"2024-09-13T09:25:53.166440+0000","last_became_peered":"2024-09-13T09:25:53.166440+0000","last_unstale":"2024-09-13T09:26:16.565826+0000","last_undegraded":"2024-09-13T09:26:16.565826+0000","last_fullsized":"2024-09-13T09:26:16.565826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:47:23.151437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":15777693,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":203,"num_read_kb":283,"num_write":53,"num_write_kb":580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"41'43","reported_seq":287,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.437437+0000","last_change":"2024-09-13T09:25:58.216595+0000","last_active":"2024-09-13T09:26:16.437437+0000","last_peered":"2024-09-13T09:26:16.437437+0000","last_clean":"2024-09-13T09:26:16.437437+0000","last_became_active":"2024-09-13T09:25:58.202391+0000","last_became_peered":"2024-09-13T09:25:58.202391+0000","last_unstale":"2024-09-13T09:26:16.437437+0000","last_undegraded":"2024-09-13T09:26:16.437437+0000","last_fullsized":"2024-09-13T09:26:16.437437+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:06:56.591120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":717784,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":582,"num_read_kb":537,"num_write":34,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":8,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"41'162","reported_seq":197,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.442212+0000","last_change":"2024-09-13T09:25:58.211250+0000","last_active":"2024-09-13T09:26:16.442212+0000","last_peered":"2024-09-13T09:26:16.442212+0000","last_clean":"2024-09-13T09:26:16.442212+0000","last_became_active":"2024-09-13T09:25:58.210896+0000","last_became_peered":"2024-09-13T09:25:58.210896+0000","last_unstale":"2024-09-13T09:26:16.442212+0000","last_undegraded":"2024-09-13T09:26:16.442212+0000","last_fullsized":"2024-09-13T09:26:16.442212+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:29:33.397561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":704,"num_read_kb":548,"num_write":248,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"41'96","reported_seq":300,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.407902+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:26:16.407902+0000","last_peered":"2024-09-13T09:26:16.407902+0000","last_clean":"2024-09-13T09:26:16.407902+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:26:16.407902+0000","last_undegraded":"2024-09-13T09:26:16.407902+0000","last_fullsized":"2024-09-13T09:26:16.407902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":23429155,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":494,"num_read_kb":6049,"num_write":129,"num_write_kb":1507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"41'23","reported_seq":68,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.387762+0000","last_change":"2024-09-13T09:25:55.218654+0000","last_active":"2024-09-13T09:26:16.387762+0000","last_peered":"2024-09-13T09:26:16.387762+0000","last_clean":"2024-09-13T09:26:16.387762+0000","last_became_active":"2024-09-13T09:25:53.176649+0000","last_became_peered":"2024-09-13T09:25:53.176649+0000","last_unstale":"2024-09-13T09:26:16.387762+0000","last_undegraded":"2024-09-13T09:26:16.387762+0000","last_fullsized":"2024-09-13T09:26:16.387762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:24.563411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":4616266,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":15,"num_write":22,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"41'37","reported_seq":279,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.386653+0000","last_change":"2024-09-13T09:25:55.188342+0000","last_active":"2024-09-13T09:26:16.386653+0000","last_peered":"2024-09-13T09:26:16.386653+0000","last_clean":"2024-09-13T09:26:16.386653+0000","last_became_active":"2024-09-13T09:25:53.178133+0000","last_became_peered":"2024-09-13T09:25:53.178133+0000","last_unstale":"2024-09-13T09:26:16.386653+0000","last_undegraded":"2024-09-13T09:26:16.386653+0000","last_fullsized":"2024-09-13T09:26:16.386653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:01.188801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":19419996,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":232,"num_read_kb":3836,"num_write":30,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.362222+0000","last_change":"2024-09-13T09:25:45.109543+0000","last_active":"2024-09-13T09:26:15.362222+0000","last_peered":"2024-09-13T09:26:15.362222+0000","last_clean":"2024-09-13T09:26:15.362222+0000","last_became_active":"2024-09-13T09:25:42.952362+0000","last_became_peered":"2024-09-13T09:25:42.952362+0000","last_unstale":"2024-09-13T09:26:15.362222+0000","last_undegraded":"2024-09-13T09:26:15.362222+0000","last_fullsized":"2024-09-13T09:26:15.362222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:39.065207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"38'34","reported_seq":143,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.849146+0000","last_change":"2024-09-13T09:25:55.201311+0000","last_active":"2024-09-13T09:26:15.849146+0000","last_peered":"2024-09-13T09:26:15.849146+0000","last_clean":"2024-09-13T09:26:15.849146+0000","last_became_active":"2024-09-13T09:25:53.175817+0000","last_became_peered":"2024-09-13T09:25:53.175817+0000","last_unstale":"2024-09-13T09:26:15.849146+0000","last_undegraded":"2024-09-13T09:26:15.849146+0000","last_fullsized":"2024-09-13T09:26:15.849146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:24:01.314273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":18476436,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":104,"num_read_kb":210,"num_write":30,"num_write_kb":287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.362174+0000","last_change":"2024-09-13T09:25:58.203812+0000","last_active":"2024-09-13T09:26:15.362174+0000","last_peered":"2024-09-13T09:26:15.362174+0000","last_clean":"2024-09-13T09:26:15.362174+0000","last_became_active":"2024-09-13T09:25:58.203492+0000","last_became_peered":"2024-09-13T09:25:58.203492+0000","last_unstale":"2024-09-13T09:26:15.362174+0000","last_undegraded":"2024-09-13T09:26:15.362174+0000","last_fullsized":"2024-09-13T09:26:15.362174+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:12.981969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"41'176","reported_seq":833,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.408810+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:26:16.408810+0000","last_peered":"2024-09-13T09:26:16.408810+0000","last_clean":"2024-09-13T09:26:16.408810+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:26:16.408810+0000","last_undegraded":"2024-09-13T09:26:16.408810+0000","last_fullsized":"2024-09-13T09:26:16.408810+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":262179,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2207,"num_read_kb":1938,"num_write":224,"num_write_kb":12599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.368933+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:26:15.368933+0000","last_peered":"2024-09-13T09:26:15.368933+0000","last_clean":"2024-09-13T09:26:15.368933+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:26:15.368933+0000","last_undegraded":"2024-09-13T09:26:15.368933+0000","last_fullsized":"2024-09-13T09:26:15.368933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"41'41","reported_seq":151,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.367473+0000","last_change":"2024-09-13T09:25:55.208924+0000","last_active":"2024-09-13T09:26:16.367473+0000","last_peered":"2024-09-13T09:26:16.367473+0000","last_clean":"2024-09-13T09:26:16.367473+0000","last_became_active":"2024-09-13T09:25:53.171139+0000","last_became_peered":"2024-09-13T09:25:53.171139+0000","last_unstale":"2024-09-13T09:26:16.367473+0000","last_undegraded":"2024-09-13T09:26:16.367473+0000","last_fullsized":"2024-09-13T09:26:16.367473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:51.468156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":20242718,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":94,"num_read_kb":187,"num_write":45,"num_write_kb":366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.358356+0000","last_change":"2024-09-13T09:25:44.961992+0000","last_active":"2024-09-13T09:26:15.358356+0000","last_peered":"2024-09-13T09:26:15.358356+0000","last_clean":"2024-09-13T09:26:15.358356+0000","last_became_active":"2024-09-13T09:25:42.949161+0000","last_became_peered":"2024-09-13T09:25:42.949161+0000","last_unstale":"2024-09-13T09:26:15.358356+0000","last_undegraded":"2024-09-13T09:26:15.358356+0000","last_fullsized":"2024-09-13T09:26:15.358356+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:12:44.025441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077301900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"41'71","reported_seq":514,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.436321+0000","last_change":"2024-09-13T09:25:58.209367+0000","last_active":"2024-09-13T09:26:16.436321+0000","last_peered":"2024-09-13T09:26:16.436321+0000","last_clean":"2024-09-13T09:26:16.436321+0000","last_became_active":"2024-09-13T09:25:58.209013+0000","last_became_peered":"2024-09-13T09:25:58.209013+0000","last_unstale":"2024-09-13T09:26:16.436321+0000","last_undegraded":"2024-09-13T09:26:16.436321+0000","last_fullsized":"2024-09-13T09:26:16.436321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:58:01.150567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":832094,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1244,"num_read_kb":1090,"num_write":64,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"40'57","reported_seq":416,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.319717+0000","last_change":"2024-09-13T09:25:58.209033+0000","last_active":"2024-09-13T09:26:16.319717+0000","last_peered":"2024-09-13T09:26:16.319717+0000","last_clean":"2024-09-13T09:26:16.319717+0000","last_became_active":"2024-09-13T09:25:58.208751+0000","last_became_peered":"2024-09-13T09:25:58.208751+0000","last_unstale":"2024-09-13T09:26:16.319717+0000","last_undegraded":"2024-09-13T09:26:16.319717+0000","last_fullsized":"2024-09-13T09:26:16.319717+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.999004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1019,"num_read_kb":862,"num_write":52,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":181,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.430577+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:26:15.430577+0000","last_peered":"2024-09-13T09:26:15.430577+0000","last_clean":"2024-09-13T09:26:15.430577+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:26:15.430577+0000","last_undegraded":"2024-09-13T09:26:15.430577+0000","last_fullsized":"2024-09-13T09:26:15.430577+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":39,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.472927+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:26:15.472927+0000","last_peered":"2024-09-13T09:26:15.472927+0000","last_clean":"2024-09-13T09:26:15.472927+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:26:15.472927+0000","last_undegraded":"2024-09-13T09:26:15.472927+0000","last_fullsized":"2024-09-13T09:26:15.472927+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"41'89","reported_seq":454,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.408411+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:26:16.408411+0000","last_peered":"2024-09-13T09:26:16.408411+0000","last_clean":"2024-09-13T09:26:16.408411+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:26:16.408411+0000","last_undegraded":"2024-09-13T09:26:16.408411+0000","last_fullsized":"2024-09-13T09:26:16.408411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":19737358,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":875,"num_read_kb":801,"num_write":80,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.430709+0000","last_change":"2024-09-13T09:25:58.215642+0000","last_active":"2024-09-13T09:26:15.430709+0000","last_peered":"2024-09-13T09:26:15.430709+0000","last_clean":"2024-09-13T09:26:15.430709+0000","last_became_active":"2024-09-13T09:25:58.211664+0000","last_became_peered":"2024-09-13T09:25:58.211664+0000","last_unstale":"2024-09-13T09:26:15.430709+0000","last_undegraded":"2024-09-13T09:26:15.430709+0000","last_fullsized":"2024-09-13T09:26:15.430709+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:40.364839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"41'59","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.557702+0000","last_change":"2024-09-13T09:25:58.212397+0000","last_active":"2024-09-13T09:26:16.557702+0000","last_peered":"2024-09-13T09:26:16.557702+0000","last_clean":"2024-09-13T09:26:16.557702+0000","last_became_active":"2024-09-13T09:25:58.212107+0000","last_became_peered":"2024-09-13T09:25:58.212107+0000","last_unstale":"2024-09-13T09:26:16.557702+0000","last_undegraded":"2024-09-13T09:26:16.557702+0000","last_fullsized":"2024-09-13T09:26:16.557702+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:52:37.963201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":32254173,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":109,"num_read_kb":2506,"num_write":63,"num_write_kb":1101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":46,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.373904+0000","last_change":"2024-09-13T09:25:58.210139+0000","last_active":"2024-09-13T09:26:15.373904+0000","last_peered":"2024-09-13T09:26:15.373904+0000","last_clean":"2024-09-13T09:26:15.373904+0000","last_became_active":"2024-09-13T09:25:58.209840+0000","last_became_peered":"2024-09-13T09:25:58.209840+0000","last_unstale":"2024-09-13T09:26:15.373904+0000","last_undegraded":"2024-09-13T09:26:15.373904+0000","last_fullsized":"2024-09-13T09:26:15.373904+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:20:09.660609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"40'26","reported_seq":108,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.318354+0000","last_change":"2024-09-13T09:25:55.209051+0000","last_active":"2024-09-13T09:26:16.318354+0000","last_peered":"2024-09-13T09:26:16.318354+0000","last_clean":"2024-09-13T09:26:16.318354+0000","last_became_active":"2024-09-13T09:25:53.177322+0000","last_became_peered":"2024-09-13T09:25:53.177322+0000","last_unstale":"2024-09-13T09:26:16.318354+0000","last_undegraded":"2024-09-13T09:26:16.318354+0000","last_fullsized":"2024-09-13T09:26:16.318354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:04.763964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":3953922,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":71,"num_read_kb":107,"num_write":23,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.430637+0000","last_change":"2024-09-13T09:25:58.204165+0000","last_active":"2024-09-13T09:26:15.430637+0000","last_peered":"2024-09-13T09:26:15.430637+0000","last_clean":"2024-09-13T09:26:15.430637+0000","last_became_active":"2024-09-13T09:25:58.203817+0000","last_became_peered":"2024-09-13T09:25:58.203817+0000","last_unstale":"2024-09-13T09:26:15.430637+0000","last_undegraded":"2024-09-13T09:26:15.430637+0000","last_fullsized":"2024-09-13T09:26:15.430637+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:50.258811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"41'99","reported_seq":181,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.431577+0000","last_change":"2024-09-13T09:25:55.208255+0000","last_active":"2024-09-13T09:26:16.431577+0000","last_peered":"2024-09-13T09:26:16.431577+0000","last_clean":"2024-09-13T09:26:16.431577+0000","last_became_active":"2024-09-13T09:25:53.167023+0000","last_became_peered":"2024-09-13T09:25:53.167023+0000","last_unstale":"2024-09-13T09:26:16.431577+0000","last_undegraded":"2024-09-13T09:26:16.431577+0000","last_fullsized":"2024-09-13T09:26:16.431577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:15.699046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":3450451,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":382,"num_read_kb":461,"num_write":137,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"41'53","reported_seq":130,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.444907+0000","last_change":"2024-09-13T09:25:55.187602+0000","last_active":"2024-09-13T09:26:16.444907+0000","last_peered":"2024-09-13T09:26:16.444907+0000","last_clean":"2024-09-13T09:26:16.444907+0000","last_became_active":"2024-09-13T09:25:53.646396+0000","last_became_peered":"2024-09-13T09:25:53.646396+0000","last_unstale":"2024-09-13T09:26:16.444907+0000","last_undegraded":"2024-09-13T09:26:16.444907+0000","last_fullsized":"2024-09-13T09:26:16.444907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:21.102946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":18851621,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":67,"num_read_kb":266,"num_write":50,"num_write_kb":392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"40'22","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:15.562483+0000","last_change":"2024-09-13T09:25:55.752827+0000","last_active":"2024-09-13T09:26:15.562483+0000","last_peered":"2024-09-13T09:26:15.562483+0000","last_clean":"2024-09-13T09:26:15.562483+0000","last_became_active":"2024-09-13T09:25:53.180652+0000","last_became_peered":"2024-09-13T09:25:53.180652+0000","last_unstale":"2024-09-13T09:26:15.562483+0000","last_undegraded":"2024-09-13T09:26:15.562483+0000","last_fullsized":"2024-09-13T09:26:15.562483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:31.395047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":4390912,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":21,"num_read_kb":153,"num_write":21,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"41'64","reported_seq":247,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.434032+0000","last_change":"2024-09-13T09:25:58.204397+0000","last_active":"2024-09-13T09:26:16.434032+0000","last_peered":"2024-09-13T09:26:16.434032+0000","last_clean":"2024-09-13T09:26:16.434032+0000","last_became_active":"2024-09-13T09:25:58.204077+0000","last_became_peered":"2024-09-13T09:25:58.204077+0000","last_unstale":"2024-09-13T09:26:16.434032+0000","last_undegraded":"2024-09-13T09:26:16.434032+0000","last_fullsized":"2024-09-13T09:26:16.434032+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:52:46.564832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":41276601,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":187,"num_read_kb":6154,"num_write":78,"num_write_kb":13736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"41'241","reported_seq":345,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.379070+0000","last_change":"2024-09-13T09:25:55.199660+0000","last_active":"2024-09-13T09:26:16.379070+0000","last_peered":"2024-09-13T09:26:16.379070+0000","last_clean":"2024-09-13T09:26:16.379070+0000","last_became_active":"2024-09-13T09:25:53.179342+0000","last_became_peered":"2024-09-13T09:25:53.179342+0000","last_unstale":"2024-09-13T09:26:16.379070+0000","last_undegraded":"2024-09-13T09:26:16.379070+0000","last_fullsized":"2024-09-13T09:26:16.379070+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:27:50.835430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":16646476,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1101,"num_read_kb":1225,"num_write":386,"num_write_kb":856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"41'52","reported_seq":112,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.387760+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:26:16.387760+0000","last_peered":"2024-09-13T09:26:16.387760+0000","last_clean":"2024-09-13T09:26:16.387760+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:26:16.387760+0000","last_undegraded":"2024-09-13T09:26:16.387760+0000","last_fullsized":"2024-09-13T09:26:16.387760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":6284030,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":55,"num_read_kb":2188,"num_write":63,"num_write_kb":12809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"41'41","reported_seq":104,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.440527+0000","last_change":"2024-09-13T09:25:55.220725+0000","last_active":"2024-09-13T09:26:16.440527+0000","last_peered":"2024-09-13T09:26:16.440527+0000","last_clean":"2024-09-13T09:26:16.440527+0000","last_became_active":"2024-09-13T09:25:53.178164+0000","last_became_peered":"2024-09-13T09:25:53.178164+0000","last_unstale":"2024-09-13T09:26:16.440527+0000","last_undegraded":"2024-09-13T09:26:16.440527+0000","last_fullsized":"2024-09-13T09:26:16.440527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:52.167054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":13217289,"num_objects":13,"num_object_clones":6,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":56,"num_read_kb":603,"num_write":32,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"41'43","reported_seq":116,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.434821+0000","last_change":"2024-09-13T09:25:55.219599+0000","last_active":"2024-09-13T09:26:16.434821+0000","last_peered":"2024-09-13T09:26:16.434821+0000","last_clean":"2024-09-13T09:26:16.434821+0000","last_became_active":"2024-09-13T09:25:53.177574+0000","last_became_peered":"2024-09-13T09:25:53.177574+0000","last_unstale":"2024-09-13T09:26:16.434821+0000","last_undegraded":"2024-09-13T09:26:16.434821+0000","last_fullsized":"2024-09-13T09:26:16.434821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:35.248811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":5411450,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":81,"num_read_kb":145,"num_write":36,"num_write_kb":403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"41'92","reported_seq":161,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.558850+0000","last_change":"2024-09-13T09:25:58.206682+0000","last_active":"2024-09-13T09:26:16.558850+0000","last_peered":"2024-09-13T09:26:16.558850+0000","last_clean":"2024-09-13T09:26:16.558850+0000","last_became_active":"2024-09-13T09:25:58.206323+0000","last_became_peered":"2024-09-13T09:25:58.206323+0000","last_unstale":"2024-09-13T09:26:16.558850+0000","last_undegraded":"2024-09-13T09:26:16.558850+0000","last_fullsized":"2024-09-13T09:26:16.558850+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:11:46.537802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":10854705,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":415,"num_read_kb":820,"num_write":149,"num_write_kb":324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"40'65","reported_seq":1130,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.263331+0000","last_change":"2024-09-13T09:25:55.219871+0000","last_active":"2024-09-13T09:26:16.263331+0000","last_peered":"2024-09-13T09:26:16.263331+0000","last_clean":"2024-09-13T09:26:16.263331+0000","last_became_active":"2024-09-13T09:25:53.179182+0000","last_became_peered":"2024-09-13T09:25:53.179182+0000","last_unstale":"2024-09-13T09:26:16.263331+0000","last_undegraded":"2024-09-13T09:26:16.263331+0000","last_fullsized":"2024-09-13T09:26:16.263331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:45.198757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":15431740,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1148,"num_read_kb":3741,"num_write":69,"num_write_kb":2040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"41'72","reported_seq":383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.471989+0000","last_change":"2024-09-13T09:25:58.209400+0000","last_active":"2024-09-13T09:26:16.471989+0000","last_peered":"2024-09-13T09:26:16.471989+0000","last_clean":"2024-09-13T09:26:16.471989+0000","last_became_active":"2024-09-13T09:25:58.209004+0000","last_became_peered":"2024-09-13T09:25:58.209004+0000","last_unstale":"2024-09-13T09:26:16.471989+0000","last_undegraded":"2024-09-13T09:26:16.471989+0000","last_fullsized":"2024-09-13T09:26:16.471989+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:08:25.744656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":748236,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":950,"num_read_kb":854,"num_write":84,"num_write_kb":205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"41'98","reported_seq":578,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:26:16.443522+0000","last_change":"2024-09-13T09:25:55.219148+0000","last_active":"2024-09-13T09:26:16.443522+0000","last_peered":"2024-09-13T09:26:16.443522+0000","last_clean":"2024-09-13T09:26:16.443522+0000","last_became_active":"2024-09-13T09:25:53.177434+0000","last_became_peered":"2024-09-13T09:25:53.177434+0000","last_unstale":"2024-09-13T09:26:16.443522+0000","last_undegraded":"2024-09-13T09:26:16.443522+0000","last_fullsized":"2024-09-13T09:26:16.443522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:13:59.794157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":22591500,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":1094,"num_read_kb":1033,"num_write":89,"num_write_kb":632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":398410353,"num_objects":270,"num_object_clones":70,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":64,"num_read":17037,"num_read_kb":45795,"num_write":2790,"num_write_kb":53220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":16,"num_keys_recovered":8,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6057984,"data_stored":32062046,"data_compressed":1308692,"data_compressed_allocated":1261568,"data_compressed_original":27639724,"omap_allocated":0,"internal_metadata":0},"log_size":2311,"ondisk_log_size":2311,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33168,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338672,"statfs":{"total":96636764160,"available":96602800128,"internally_reserved":0,"allocated":2105344,"data_stored":9614640,"data_compressed":385940,"data_compressed_allocated":372736,"data_compressed_original":8052736,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":798720,"data_stored":1183626,"data_compressed":34431,"data_compressed_allocated":45056,"data_compressed_original":729088,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509452,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1359872,"data_stored":5106231,"data_compressed":205384,"data_compressed_allocated":200704,"data_compressed_original":4308950,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2174976,"data_stored":10446044,"data_compressed":423520,"data_compressed_allocated":393216,"data_compressed_original":8970240,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1351680,"data_stored":6019298,"data_compressed":250548,"data_compressed_allocated":253952,"data_compressed_original":5304320,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":921600,"data_stored":1653634,"data_compressed":62693,"data_compressed_allocated":118784,"data_compressed_original":1200128,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]}]},{"osd":0,"up_from":0,"seq":64424509449,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":696320,"data_stored":1017355,"data_compressed":33650,"data_compressed_allocated":49152,"data_compressed_original":671702,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":16,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":628372,"data_compressed":18956,"data_compressed_allocated":32768,"data_compressed_original":401408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":838481,"data_compressed":33650,"data_compressed_allocated":49152,"data_compressed_original":671702,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":913408,"data_stored":5835196,"data_compressed":250548,"data_compressed_allocated":253952,"data_compressed_original":5304320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1728512,"data_stored":10260987,"data_compressed":423520,"data_compressed_allocated":393216,"data_compressed_original":8970240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":921600,"data_stored":4922102,"data_compressed":205384,"data_compressed_allocated":200704,"data_compressed_original":4308950,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":998561,"data_compressed":34431,"data_compressed_allocated":45056,"data_compressed_original":729088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1515520,"data_stored":8578331,"data_compressed":342203,"data_compressed_allocated":286720,"data_compressed_original":7254016,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:26:20.178 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:26:20.178 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:20.178 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-13T09:26:20.178 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:26:20.179 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:26:20.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-13T09:26:20.179 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.179 INFO:teuthology.orchestra.run.smithi063.stdout:1274 read 0x34a6261 thru 0x34a8a88 (0x2828 bytes) 2024-09-13T09:26:20.179 INFO:teuthology.orchestra.run.smithi063.stdout:1276 writesame 0xa6742fa thru 0xa6783ca data_size 0x1(0x40d1 bytes) 2024-09-13T09:26:20.193 INFO:teuthology.orchestra.run.smithi063.stdout:1277 write 0xbb88f4 thru 0xbbced7 (0x45e4 bytes) 2024-09-13T09:26:20.199 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.201 INFO:teuthology.orchestra.run.smithi063.stdout:1279 trunc from 0xbe7308c to 0xcd56961 2024-09-13T09:26:20.203 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:20.199+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:20.205 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.205 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.214 INFO:teuthology.orchestra.run.smithi063.stdout:1280 write 0x21df67a thru 0x21e13c9 (0x1d50 bytes) 2024-09-13T09:26:20.215 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:20.220 INFO:teuthology.orchestra.run.smithi063.stdout:1282 read 0x342e536 thru 0x343112a (0x2bf5 bytes) 2024-09-13T09:26:20.221 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.221 INFO:teuthology.orchestra.run.smithi063.stdout:1284 read 0x37f089d thru 0x37fc173 (0xb8d7 bytes) 2024-09-13T09:26:20.221 INFO:teuthology.orchestra.run.smithi063.stdout:1286 read 0x6343bb5 thru 0x634e9d9 (0xae25 bytes) 2024-09-13T09:26:20.221 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.222 INFO:teuthology.orchestra.run.smithi063.stdout:1290 trunc from 0xcd56961 to 0xda351cc 2024-09-13T09:26:20.234 INFO:teuthology.orchestra.run.smithi063.stdout:1293 write 0x96daa7e thru 0x96e9d65 (0xf2e8 bytes) 2024-09-13T09:26:20.240 INFO:teuthology.orchestra.run.smithi063.stdout:1294 write 0x1adc1f4 thru 0x1ae65ee (0xa3fb bytes) 2024-09-13T09:26:20.247 INFO:teuthology.orchestra.run.smithi063.stdout:1296 read 0x35ad27a thru 0x35b8ea2 (0xbc29 bytes) 2024-09-13T09:26:20.247 INFO:teuthology.orchestra.run.smithi063.stdout:1297 write 0xb8345fd thru 0xb8413e2 (0xcde6 bytes) 2024-09-13T09:26:20.256 INFO:teuthology.orchestra.run.smithi063.stdout:1298 read 0x4ace937 thru 0x4ad60cc (0x7796 bytes) 2024-09-13T09:26:20.256 INFO:teuthology.orchestra.run.smithi063.stdout:1299 write 0xd2c634d thru 0xd2cb9f5 (0x56a9 bytes) 2024-09-13T09:26:20.263 INFO:teuthology.orchestra.run.smithi063.stdout:1300 punch from 0xba49c1f to 0xba5882c, (0xec0d bytes) 2024-09-13T09:26:20.263 INFO:teuthology.orchestra.run.smithi063.stdout:1301 write 0x6dd3221 thru 0x6ddf074 (0xbe54 bytes) 2024-09-13T09:26:20.269 INFO:teuthology.orchestra.run.smithi063.stdout:1302 punch from 0xb7e792d to 0xb7e7d3b, (0x40e bytes) 2024-09-13T09:26:20.269 INFO:teuthology.orchestra.run.smithi063.stdout:1303 writesame 0xd8d4fd3 thru 0xd8d7c01 data_size 0x1(0x2c2f bytes) 2024-09-13T09:26:20.281 INFO:teuthology.orchestra.run.smithi063.stdout:1305 trunc from 0xda351cc to 0xda2ead4 2024-09-13T09:26:20.292 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.292 INFO:teuthology.orchestra.run.smithi063.stdout:1307 write 0x547c57c thru 0x5480852 (0x42d7 bytes) 2024-09-13T09:26:20.299 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.300 INFO:teuthology.orchestra.run.smithi063.stdout:1309 read 0xab2b4a3 thru 0xab2fcf4 (0x4852 bytes) 2024-09-13T09:26:20.301 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:26:20.301 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:20.304 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:20.299+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:20.319 INFO:teuthology.orchestra.run.smithi063.stdout:1310 trunc from 0xda2ead4 to 0x3f2d9a1 2024-09-13T09:26:20.321 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.321 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.333 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:20.338 INFO:teuthology.orchestra.run.smithi063.stdout:1311 punch from 0x1319329 to 0x13283b5, (0xf08c bytes) 2024-09-13T09:26:20.339 INFO:teuthology.orchestra.run.smithi063.stdout:1312 punch from 0x179e69c to 0x179fd26, (0x168a bytes) 2024-09-13T09:26:20.339 INFO:teuthology.orchestra.run.smithi063.stdout:1313 read 0x2ce4327 thru 0x2ce71c1 (0x2e9b bytes) 2024-09-13T09:26:20.355 INFO:teuthology.orchestra.run.smithi063.stdout:1314 write 0x82ddc04 thru 0x82e01b5 (0x25b2 bytes) 2024-09-13T09:26:20.361 INFO:teuthology.orchestra.run.smithi063.stdout:1315 write 0x646cbee thru 0x646e3a7 (0x17ba bytes) 2024-09-13T09:26:20.366 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.366 INFO:teuthology.orchestra.run.smithi063.stdout:1319 read 0x4ad54a4 thru 0x4adc979 (0x74d6 bytes) 2024-09-13T09:26:20.368 INFO:teuthology.orchestra.run.smithi063.stdout:1320 trunc from 0x82e01b6 to 0x7460bfd 2024-09-13T09:26:20.389 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.389 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.389 INFO:teuthology.orchestra.run.smithi063.stdout:1325 read 0x140bd60 thru 0x140bded (0x8e bytes) 2024-09-13T09:26:20.405 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:20.401+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:20.415 INFO:teuthology.orchestra.run.smithi063.stdout:1326 writesame 0xe9fb9f7 thru 0xea0329f data_size 0x17(0x78a9 bytes) 2024-09-13T09:26:20.416 INFO:teuthology.orchestra.run.smithi063.stdout:1327 read 0x114e614 thru 0x114fbab (0x1598 bytes) 2024-09-13T09:26:20.416 INFO:teuthology.orchestra.run.smithi063.stdout:1328 writesame 0x237db78 thru 0x2385c2f data_size 0xc(0x80b8 bytes) 2024-09-13T09:26:20.425 INFO:teuthology.orchestra.run.smithi063.stdout:1329 punch from 0x2de08cc to 0x2de8d16, (0x844a bytes) 2024-09-13T09:26:20.441 INFO:teuthology.orchestra.run.smithi063.stdout:1331 trunc from 0xea032a0 to 0x5447f0c 2024-09-13T09:26:20.453 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.453 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.464 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:20.464 INFO:teuthology.orchestra.run.smithi063.stdout:1335 writesame 0x37e9003 thru 0x37ef504 data_size 0x7(0x6502 bytes) 2024-09-13T09:26:20.472 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.472 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.495 INFO:teuthology.orchestra.run.smithi063.stdout:1343 write 0x98a6990 thru 0x98b601d (0xf68e bytes) 2024-09-13T09:26:20.501 INFO:teuthology.orchestra.run.smithi063.stdout:1344 read 0x59985ef thru 0x59a53c1 (0xcdd3 bytes) 2024-09-13T09:26:20.506 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:20.503+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:20.515 INFO:teuthology.orchestra.run.smithi063.stdout:1345 write 0xa496f2b thru 0xa4a05a0 (0x9676 bytes) 2024-09-13T09:26:20.527 INFO:teuthology.orchestra.run.smithi063.stdout:1346 trunc from 0xa4a05a1 to 0xd5de399 2024-09-13T09:26:20.540 INFO:teuthology.orchestra.run.smithi063.stdout:1347 read 0x4d36703 thru 0x4d3c776 (0x6074 bytes) 2024-09-13T09:26:20.540 INFO:teuthology.orchestra.run.smithi063.stdout:1349 read 0x1bf5afa thru 0x1c007fc (0xad03 bytes) 2024-09-13T09:26:20.541 INFO:teuthology.orchestra.run.smithi063.stdout:1350 read 0xb9ae43d thru 0xb9b06ff (0x22c3 bytes) 2024-09-13T09:26:20.541 INFO:teuthology.orchestra.run.smithi063.stdout:1353 write 0x6fbccbd thru 0x6fc4f9c (0x82e0 bytes) 2024-09-13T09:26:20.550 INFO:teuthology.orchestra.run.smithi063.stdout:1354 read 0xd45eab5 thru 0xd464733 (0x5c7f bytes) 2024-09-13T09:26:20.550 INFO:teuthology.orchestra.run.smithi063.stdout:1355 write 0x3684c46 thru 0x36850d5 (0x490 bytes) 2024-09-13T09:26:20.555 INFO:teuthology.orchestra.run.smithi063.stdout:1356 write 0x27c1cb9 thru 0x27c8263 (0x65ab bytes) 2024-09-13T09:26:20.558 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.573 INFO:teuthology.orchestra.run.smithi063.stdout:1361 trunc from 0xd5de399 to 0x361c908 2024-09-13T09:26:20.588 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.588 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.594 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:20.606 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:20.605+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:20.611 INFO:teuthology.orchestra.run.smithi063.stdout:1363 writesame 0x721f2aa thru 0x722da17 data_size 0xb(0xe76e bytes) 2024-09-13T09:26:20.622 INFO:teuthology.orchestra.run.smithi063.stdout:1364 trunc from 0x722da18 to 0xc39874d 2024-09-13T09:26:20.644 INFO:teuthology.orchestra.run.smithi063.stdout:1365 trunc from 0xc39874d to 0x6d24cdc 2024-09-13T09:26:20.658 INFO:teuthology.orchestra.run.smithi063.stdout:1366 punch from 0x1f011d8 to 0x1f1069c, (0xf4c4 bytes) 2024-09-13T09:26:20.658 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.658 INFO:teuthology.orchestra.run.smithi063.stdout:1368 read 0x322c56a thru 0x3233320 (0x6db7 bytes) 2024-09-13T09:26:20.674 INFO:teuthology.orchestra.run.smithi063.stdout:1369 write 0x7c5f138 thru 0x7c6724a (0x8113 bytes) 2024-09-13T09:26:20.681 INFO:teuthology.orchestra.run.smithi063.stdout:1370 write 0x649c58d thru 0x64aaed4 (0xe948 bytes) 2024-09-13T09:26:20.689 INFO:teuthology.orchestra.run.smithi063.stdout:1371 write 0x6b014ec thru 0x6b10e90 (0xf9a5 bytes) 2024-09-13T09:26:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:1373 read 0x57a6ccc thru 0x57b1e5d (0xb192 bytes) 2024-09-13T09:26:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.695 INFO:teuthology.orchestra.run.smithi063.stdout:1380 write 0x70eff86 thru 0x70f703d (0x70b8 bytes) 2024-09-13T09:26:20.701 INFO:teuthology.orchestra.run.smithi063.stdout:1381 read 0x7690661 thru 0x7690a94 (0x434 bytes) 2024-09-13T09:26:20.701 INFO:teuthology.orchestra.run.smithi063.stdout:1382 write 0x365f8ed thru 0x36699cb (0xa0df bytes) 2024-09-13T09:26:20.707 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:20.705+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:20.725 INFO:teuthology.orchestra.run.smithi063.stdout:1383 trunc from 0x7c6724b to 0x1298e2f 2024-09-13T09:26:20.725 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.725 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.736 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:20.759 INFO:teuthology.orchestra.run.smithi063.stdout:1384 trunc from 0x1298e2f to 0x75fb5ef 2024-09-13T09:26:20.771 INFO:teuthology.orchestra.run.smithi063.stdout:1386 write 0x289e8 thru 0x33488 (0xaaa1 bytes) 2024-09-13T09:26:20.799 INFO:teuthology.orchestra.run.smithi063.stdout:1387 write 0xb2231be thru 0xb22c769 (0x95ac bytes) 2024-09-13T09:26:20.810 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:20.807+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:20.824 INFO:teuthology.orchestra.run.smithi063.stdout:1389 trunc from 0xb22c76a to 0x1f77089 2024-09-13T09:26:20.844 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.845 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.845 INFO:teuthology.orchestra.run.smithi063.stdout:1394 read 0x156d490 thru 0x157b6c4 (0xe235 bytes) 2024-09-13T09:26:20.874 INFO:teuthology.orchestra.run.smithi063.stdout:1396 write 0x9e5506a thru 0x9e61327 (0xc2be bytes) 2024-09-13T09:26:20.880 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.880 INFO:teuthology.orchestra.run.smithi063.stdout:1400 read 0x4154d66 thru 0x415fa29 (0xacc4 bytes) 2024-09-13T09:26:20.882 INFO:teuthology.orchestra.run.smithi063.stdout:1404 trunc from 0x9e61328 to 0xb19add6 2024-09-13T09:26:20.893 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:20.894 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:20.904 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:20.910 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:20.909+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:20.913 INFO:teuthology.orchestra.run.smithi063.stdout:1405 write 0xdf0ffcc thru 0xdf1da6b (0xdaa0 bytes) 2024-09-13T09:26:20.920 INFO:teuthology.orchestra.run.smithi063.stdout:1406 read 0xcdb8c4 thru 0xce58cc (0xa009 bytes) 2024-09-13T09:26:20.920 INFO:teuthology.orchestra.run.smithi063.stdout:1408 read 0xcaf9467 thru 0xcb002ac (0x6e46 bytes) 2024-09-13T09:26:20.920 INFO:teuthology.orchestra.run.smithi063.stdout:1409 writesame 0x68898e8 thru 0x6897c35 data_size 0x73(0xe34e bytes) 2024-09-13T09:26:20.939 INFO:teuthology.orchestra.run.smithi063.stdout:1410 write 0xe81f03c thru 0xe829ea6 (0xae6b bytes) 2024-09-13T09:26:20.946 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:20.946 INFO:teuthology.orchestra.run.smithi063.stdout:1412 punch from 0x26677d9 to 0x266e3b0, (0x6bd7 bytes) 2024-09-13T09:26:20.946 INFO:teuthology.orchestra.run.smithi063.stdout:1414 write 0x3f8747c thru 0x3f885f8 (0x117d bytes) 2024-09-13T09:26:20.977 INFO:teuthology.orchestra.run.smithi063.stdout:1418 trunc from 0xe829ea7 to 0x158ca0e 2024-09-13T09:26:21.011 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:21.009+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:21.012 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.012 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.018 INFO:teuthology.orchestra.run.smithi063.stdout:1419 write 0x69871d1 thru 0x699183c (0xa66c bytes) 2024-09-13T09:26:21.022 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:21.023 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.042 INFO:teuthology.orchestra.run.smithi063.stdout:1422 write 0x96cd5fa thru 0x96d31b8 (0x5bbf bytes) 2024-09-13T09:26:21.067 INFO:teuthology.orchestra.run.smithi063.stdout:1423 write 0xc29577f thru 0xc2a23a2 (0xcc24 bytes) 2024-09-13T09:26:21.076 INFO:teuthology.orchestra.run.smithi063.stdout:1424 punch from 0x3e8ffa to 0x3f681e, (0xd824 bytes) 2024-09-13T09:26:21.077 INFO:teuthology.orchestra.run.smithi063.stdout:1425 read 0x5e5452a thru 0x5e564a0 (0x1f77 bytes) 2024-09-13T09:26:21.077 INFO:teuthology.orchestra.run.smithi063.stdout:1426 punch from 0x1fb6591 to 0x1fbe312, (0x7d81 bytes) 2024-09-13T09:26:21.077 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.077 INFO:teuthology.orchestra.run.smithi063.stdout:1428 write 0xbd292ed thru 0xbd3766e (0xe382 bytes) 2024-09-13T09:26:21.084 INFO:teuthology.orchestra.run.smithi063.stdout:1429 read 0x601db31 thru 0x6024caa (0x717a bytes) 2024-09-13T09:26:21.084 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.084 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.084 INFO:teuthology.orchestra.run.smithi063.stdout:1434 writesame 0x21e5c0d thru 0x21ec6cf data_size 0x1(0x6ac3 bytes) 2024-09-13T09:26:21.107 INFO:teuthology.orchestra.run.smithi063.stdout:1436 read 0x95cc366 thru 0x95d8705 (0xc3a0 bytes) 2024-09-13T09:26:21.107 INFO:teuthology.orchestra.run.smithi063.stdout:1437 read 0x421fc69 thru 0x42217d6 (0x1b6e bytes) 2024-09-13T09:26:21.107 INFO:teuthology.orchestra.run.smithi063.stdout:1438 write 0x51533f5 thru 0x515921f (0x5e2b bytes) 2024-09-13T09:26:21.112 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:21.107+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:21.128 INFO:teuthology.orchestra.run.smithi063.stdout:1439 write 0xdea27b5 thru 0xdea3832 (0x107e bytes) 2024-09-13T09:26:21.158 INFO:teuthology.orchestra.run.smithi063.stdout:1440 trunc from 0xdea3833 to 0x2e2b4bb 2024-09-13T09:26:21.180 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.180 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:21.195 INFO:teuthology.orchestra.run.smithi063.stdout:1441 trunc from 0x2e2b4bb to 0x9046825 2024-09-13T09:26:21.208 INFO:teuthology.orchestra.run.smithi063.stdout:1442 punch from 0x3f12bab to 0x3f2228e, (0xf6e3 bytes) 2024-09-13T09:26:21.208 INFO:teuthology.orchestra.run.smithi063.stdout:1444 read 0x35e722c thru 0x35ee94c (0x7721 bytes) 2024-09-13T09:26:21.208 INFO:teuthology.orchestra.run.smithi063.stdout:1445 read 0x491419d thru 0x4919144 (0x4fa8 bytes) 2024-09-13T09:26:21.208 INFO:teuthology.orchestra.run.smithi063.stdout:1446 read 0x35676fd thru 0x35691b9 (0x1abd bytes) 2024-09-13T09:26:21.208 INFO:teuthology.orchestra.run.smithi063.stdout:1447 writesame 0x614fb94 thru 0x615f525 data_size 0x5(0xf992 bytes) 2024-09-13T09:26:21.213 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:21.213+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:21.224 INFO:teuthology.orchestra.run.smithi063.stdout:1448 trunc from 0x9046825 to 0x99dca7a 2024-09-13T09:26:21.257 INFO:teuthology.orchestra.run.smithi063.stdout:1450 write 0xc198f59 thru 0xc1a22ea (0x9392 bytes) 2024-09-13T09:26:21.264 INFO:teuthology.orchestra.run.smithi063.stdout:1451 punch from 0x3b4389f to 0x3b4eb6f, (0xb2d0 bytes) 2024-09-13T09:26:21.264 INFO:teuthology.orchestra.run.smithi063.stdout:1452 punch from 0x13a6b79 to 0x13ad677, (0x6afe bytes) 2024-09-13T09:26:21.264 INFO:teuthology.orchestra.run.smithi063.stdout:1453 writesame 0x8c444aa thru 0x8c47eea data_size 0x3(0x3a41 bytes) 2024-09-13T09:26:21.291 INFO:teuthology.orchestra.run.smithi063.stdout:1455 writesame 0xdfab909 thru 0xdfbb777 data_size 0x7(0xfe6f bytes) 2024-09-13T09:26:21.292 INFO:teuthology.orchestra.run.smithi063.stdout:1457 write 0xc40cad4 thru 0xc4178e3 (0xae10 bytes) 2024-09-13T09:26:21.295 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.295 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.305 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:21.314 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:21.311+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:21.320 INFO:teuthology.orchestra.run.smithi063.stdout:1458 trunc from 0xdfbb778 to 0x383bc2f 2024-09-13T09:26:21.360 INFO:teuthology.orchestra.run.smithi063.stdout:1459 writesame 0x7d4f11e thru 0x7d50e0d data_size 0x8(0x1cf0 bytes) 2024-09-13T09:26:21.360 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.377 INFO:teuthology.orchestra.run.smithi063.stdout:1462 writesame 0x9cfef95 thru 0x9d04a70 data_size 0xa(0x5adc bytes) 2024-09-13T09:26:21.378 INFO:teuthology.orchestra.run.smithi063.stdout:1463 write 0x636d987 thru 0x6376509 (0x8b83 bytes) 2024-09-13T09:26:21.384 INFO:teuthology.orchestra.run.smithi063.stdout:1464 punch from 0x4904fcd to 0x490af76, (0x5fa9 bytes) 2024-09-13T09:26:21.384 INFO:teuthology.orchestra.run.smithi063.stdout:1465 read 0x7f2669c thru 0x7f26fa4 (0x909 bytes) 2024-09-13T09:26:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:1469 trunc from 0x9d04a71 to 0x190ba4b 2024-09-13T09:26:21.411 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.411 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.414 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:21.411+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:21.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:26:21.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:21.422 INFO:teuthology.orchestra.run.smithi063.stdout:1470 trunc from 0x190ba4b to 0x144e72f 2024-09-13T09:26:21.433 INFO:teuthology.orchestra.run.smithi063.stdout:1471 writesame 0x109174b thru 0x109cff2 data_size 0x98(0xb8a8 bytes) 2024-09-13T09:26:21.439 INFO:teuthology.orchestra.run.smithi063.stdout:1472 read 0x3ed989 thru 0x3f7a7b (0xa0f3 bytes) 2024-09-13T09:26:21.440 INFO:teuthology.orchestra.run.smithi063.stdout:1474 writesame 0x13f7f3c thru 0x13fbf3a data_size 0x3(0x3fff bytes) 2024-09-13T09:26:21.448 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.448 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.448 INFO:teuthology.orchestra.run.smithi063.stdout:1477 read 0xf1407e thru 0xf21d14 (0xdc97 bytes) 2024-09-13T09:26:21.448 INFO:teuthology.orchestra.run.smithi063.stdout:1478 read 0x118faf4 thru 0x11978fc (0x7e09 bytes) 2024-09-13T09:26:21.468 INFO:teuthology.orchestra.run.smithi063.stdout:1479 writesame 0x49e26a6 thru 0x49ef74b data_size 0x11(0xd0a6 bytes) 2024-09-13T09:26:21.468 INFO:teuthology.orchestra.run.smithi063.stdout:1480 punch from 0x182d347 to 0x182d571, (0x22a bytes) 2024-09-13T09:26:21.492 INFO:teuthology.orchestra.run.smithi063.stdout:1481 write 0xa58ec62 thru 0xa599231 (0xa5d0 bytes) 2024-09-13T09:26:21.503 INFO:teuthology.orchestra.run.smithi063.stdout:1485 trunc from 0xa599232 to 0x75b5061 2024-09-13T09:26:21.515 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:21.513+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:21.521 INFO:teuthology.orchestra.run.smithi063.stdout:1486 read 0x59c07cd thru 0x59cc426 (0xbc5a bytes) 2024-09-13T09:26:21.521 INFO:teuthology.orchestra.run.smithi063.stdout:1487 punch from 0x6b20cb9 to 0x6b21b21, (0xe68 bytes) 2024-09-13T09:26:21.521 INFO:teuthology.orchestra.run.smithi063.stdout:1488 write 0x18502 thru 0x216b2 (0x91b1 bytes) 2024-09-13T09:26:21.524 INFO:teuthology.orchestra.run.smithi063.stdout:1489 read 0x4a883ab thru 0x4a8cb0d (0x4763 bytes) 2024-09-13T09:26:21.525 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.525 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.535 INFO:teuthology.orchestra.run.smithi063.stdout:1490 write 0x7b2947a thru 0x7b31c57 (0x87de bytes) 2024-09-13T09:26:21.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:21.552 INFO:teuthology.orchestra.run.smithi063.stdout:1491 write 0x83f85bd thru 0x840445b (0xbe9f bytes) 2024-09-13T09:26:21.560 INFO:teuthology.orchestra.run.smithi063.stdout:1495 read 0x4ce11b7 thru 0x4cef4cb (0xe315 bytes) 2024-09-13T09:26:21.560 INFO:teuthology.orchestra.run.smithi063.stdout:1496 punch from 0x7fdff42 to 0x7fe1d7d, (0x1e3b bytes) 2024-09-13T09:26:21.561 INFO:teuthology.orchestra.run.smithi063.stdout:1497 write 0x2dc0b4f thru 0x2dcc2fb (0xb7ad bytes) 2024-09-13T09:26:21.565 INFO:teuthology.orchestra.run.smithi063.stdout:1498 read 0x5d84dac thru 0x5d90551 (0xb7a6 bytes) 2024-09-13T09:26:21.590 INFO:teuthology.orchestra.run.smithi063.stdout:1500 writesame 0xe3d13a6 thru 0xe3e10ed data_size 0x14(0xfd48 bytes) 2024-09-13T09:26:21.590 INFO:teuthology.orchestra.run.smithi063.stdout:1501 read 0x75f24db thru 0x75f51cd (0x2cf3 bytes) 2024-09-13T09:26:21.607 INFO:teuthology.orchestra.run.smithi063.stdout:1503 trunc from 0xe3e10ee to 0x442a166 2024-09-13T09:26:21.616 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:21.611+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:21.630 INFO:teuthology.orchestra.run.smithi063.stdout:1505 read 0x40ddaae thru 0x40e0e4f (0x33a2 bytes) 2024-09-13T09:26:21.641 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.642 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:21.655 INFO:teuthology.orchestra.run.smithi063.stdout:1507 write 0xcbcbf79 thru 0xcbd780e (0xb896 bytes) 2024-09-13T09:26:21.661 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.661 INFO:teuthology.orchestra.run.smithi063.stdout:1511 read 0xb257439 thru 0xb2638f5 (0xc4bd bytes) 2024-09-13T09:26:21.661 INFO:teuthology.orchestra.run.smithi063.stdout:1512 read 0x8105198 thru 0x81058f7 (0x760 bytes) 2024-09-13T09:26:21.661 INFO:teuthology.orchestra.run.smithi063.stdout:1513 writesame 0x27baf53 thru 0x27c611f data_size 0x1(0xb1cd bytes) 2024-09-13T09:26:21.685 INFO:teuthology.orchestra.run.smithi063.stdout:1514 read 0x7883762 thru 0x788fd2a (0xc5c9 bytes) 2024-09-13T09:26:21.686 INFO:teuthology.orchestra.run.smithi063.stdout:1515 writesame 0x49455d7 thru 0x495462f data_size 0x1(0xf059 bytes) 2024-09-13T09:26:21.717 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:21.717+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:21.725 INFO:teuthology.orchestra.run.smithi063.stdout:1516 read 0x60cf4dd thru 0x60df397 (0xfebb bytes) 2024-09-13T09:26:21.725 INFO:teuthology.orchestra.run.smithi063.stdout:1517 read 0x46fbfd1 thru 0x470542c (0x945c bytes) 2024-09-13T09:26:21.746 INFO:teuthology.orchestra.run.smithi063.stdout:1518 trunc from 0xcbd780f to 0x1072a81 2024-09-13T09:26:21.757 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:21.758 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.759 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:21.777 INFO:teuthology.orchestra.run.smithi063.stdout:1519 read 0x1007fdc thru 0x1008a92 (0xab7 bytes) 2024-09-13T09:26:21.777 INFO:teuthology.orchestra.run.smithi063.stdout:1520 write 0xc9cfe thru 0xcafd3 (0x12d6 bytes) 2024-09-13T09:26:21.812 INFO:teuthology.orchestra.run.smithi063.stdout:1521 write 0xe3d1d3b thru 0xe3d4f4e (0x3214 bytes) 2024-09-13T09:26:21.818 INFO:teuthology.orchestra.run.smithi063.stdout:1524 write 0x10fc511 thru 0x11055ce (0x90be bytes) 2024-09-13T09:26:21.819 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:21.815+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:21.843 INFO:teuthology.orchestra.run.smithi063.stdout:1526 trunc from 0xe3d4f4f to 0x4893819 2024-09-13T09:26:21.863 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:21.863 INFO:teuthology.orchestra.run.smithi063.stdout:1528 writesame 0x3917e61 thru 0x391b047 data_size 0x23(0x31e7 bytes) 2024-09-13T09:26:21.868 INFO:teuthology.orchestra.run.smithi063.stdout:1530 clone 12 order 25 su 262144 sc 12 2024-09-13T09:26:21.873 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.873 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:21.909 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:21.920 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:21.915+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:21.931 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:21.950 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:21.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:21.985 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:21.985 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:21.993 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:21.995 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:22.015 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:22.020 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:22.017+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:22.037 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:22.058 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:22.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:22.100 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:22.100 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.136975677000002, 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:22.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:22.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.140586222000003, 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:22.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:22.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:22.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.732012589999997, 2024-09-13T09:26:22.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014920113, 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 110 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365978+0000", 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365977+0000", 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.365987+0000", 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:26:22.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366004+0000", 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.662e-05 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366022+0000", 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7883000000000001e-05 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376091+0000", 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010069171999999999 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376246+0000", 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000155377 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.376548+0000", 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030137300000000002 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380798+0000", 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004250406 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380813+0000", 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4357e-05 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380862+0000", 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9137000000000001e-05 2024-09-13T09:26:22.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380898+0000", 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6158999999999997e-05 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.731954012000003, 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016811104, 2024-09-13T09:26:22.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 111 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366036+0000", 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366035+0000", 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366045+0000", 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0327e-05 2024-09-13T09:26:22.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366062+0000", 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6483000000000001e-05 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366074+0000", 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.377874+0000", 2024-09-13T09:26:22.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011799983999999999 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378006+0000", 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013130100000000001 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378143+0000", 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137157 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380914+0000", 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002771281 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382760+0000", 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001846253 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382810+0000", 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9483e-05 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382847+0000", 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7749000000000001e-05 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.731874310000002, 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018719919000000002, 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 112 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366116+0000", 2024-09-13T09:26:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366115+0000", 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366313+0000", 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000198156 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366331+0000", 2024-09-13T09:26:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7739000000000001e-05 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366345+0000", 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4226000000000001e-05 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378684+0000", 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012339325 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.378827+0000", 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014284700000000001 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.379116+0000", 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028930399999999999 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382675+0000", 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035590050000000001 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384568+0000", 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001892707 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384706+0000", 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013804099999999999 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384836+0000", 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012977900000000001 2024-09-13T09:26:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.731628151999999, 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01866932, 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366362+0000", 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366361+0000", 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366372+0000", 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0857e-05 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366388+0000", 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366401+0000", 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.326e-05 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380005+0000", 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013604036 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380100+0000", 2024-09-13T09:26:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4468000000000004e-05 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380211+0000", 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011084199999999999 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383662+0000", 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034510460000000001 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384905+0000", 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001243009 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384996+0000", 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1365000000000004e-05 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385032+0000", 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5528000000000003e-05 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.731575303, 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020964449999999999, 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 114 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.136 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366415+0000", 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366414+0000", 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366423+0000", 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366443+0000", 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9589000000000002e-05 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366455+0000", 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2157000000000001e-05 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.380966+0000", 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014510662000000001 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381062+0000", 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6526000000000004e-05 2024-09-13T09:26:22.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381259+0000", 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019658499999999999 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383675+0000", 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024160480000000001 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387295+0000", 2024-09-13T09:26:22.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036198020000000001 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387342+0000", 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7299000000000002e-05 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387380+0000", 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7379000000000002e-05 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.731151275999999, 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020678247, 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 115 2024-09-13T09:26:22.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366839+0000", 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.366838+0000", 2024-09-13T09:26:22.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367071+0000", 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023322800000000001 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367094+0000", 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3079999999999999e-05 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367111+0000", 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7008999999999999e-05 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381788+0000", 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014676850999999999 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.381894+0000", 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010610799999999999 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382045+0000", 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015073699999999999 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383689+0000", 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016436630000000001 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387445+0000", 2024-09-13T09:26:22.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037564 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387488+0000", 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3402000000000002e-05 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387517+0000", 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8975999999999998e-05 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.730861589, 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020557408999999999, 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367129+0000", 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367128+0000", 2024-09-13T09:26:22.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367138+0000", 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0672999999999999e-05 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367157+0000", 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8779000000000001e-05 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367174+0000", 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6464e-05 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.382926+0000", 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015752759000000002 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383077+0000", 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000150965 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383228+0000", 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015101399999999999 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384458+0000", 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012294370000000001 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387599+0000", 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031411960000000002 2024-09-13T09:26:22.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387647+0000", 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7648999999999997e-05 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.387686+0000", 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9653000000000001e-05 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.730802484000002, 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022933500999999998, 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367188+0000", 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367187+0000", 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367208+0000", 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1793e-05 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367225+0000", 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6668999999999999e-05 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.367239+0000", 2024-09-13T09:26:22.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4005999999999999e-05 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383748+0000", 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016508719000000002 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.383933+0000", 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018481500000000001 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.384122+0000", 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000189626 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.385049+0000", 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00092678599999999997 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390003+0000", 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049541819999999997 2024-09-13T09:26:22.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390075+0000", 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2094999999999993e-05 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.390121+0000", 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6082999999999999e-05 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140747103999999, 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015204896000000001, 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 135 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957243+0000", 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957242+0000", 2024-09-13T09:26:22.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957250+0000", 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0330000000000008e-06 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957268+0000", 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7459999999999999e-05 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957277+0000", 2024-09-13T09:26:22.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3559999999999999e-06 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965456+0000", 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081792359999999995 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965567+0000", 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110877 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.965741+0000", 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017349600000000001 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967304+0000", 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001563444 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972380+0000", 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050761019999999999 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972425+0000", 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4366000000000002e-05 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972448+0000", 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3455000000000002e-05 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.165 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140703033999998, 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015208101, 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 136 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957287+0000", 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957297+0000", 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9909999999999997e-06 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957339+0000", 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2267e-05 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957371+0000", 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2518000000000003e-05 2024-09-13T09:26:22.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966255+0000", 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088832679999999997 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966353+0000", 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8658000000000003e-05 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.966486+0000", 2024-09-13T09:26:22.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013309500000000001 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969257+0000", 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027707209999999999 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972453+0000", 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031962789999999998 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972478+0000", 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4655e-05 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972495+0000", 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7407000000000001e-05 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140588172999998, 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015156871000000001, 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957402+0000", 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957401+0000", 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957509+0000", 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010808 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957536+0000", 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6237e-05 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957542+0000", 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2020000000000001e-06 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967352+0000", 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098099580000000006 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967447+0000", 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5375000000000001e-05 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.967584+0000", 2024-09-13T09:26:22.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000136463 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971541+0000", 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039574930000000003 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972499+0000", 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095777799999999995 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972527+0000", 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7900999999999999e-05 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972559+0000", 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2191999999999999e-05 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.177 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140438336000003, 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015104328, 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957552+0000", 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957551+0000", 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957556+0000", 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.318e-06 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957565+0000", 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.115e-06 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957572+0000", 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968300+0000", 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010727617 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968387+0000", 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7021000000000006e-05 2024-09-13T09:26:22.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.968482+0000", 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5081999999999995e-05 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971595+0000", 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031126639999999998 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972588+0000", 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099296600000000007 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972635+0000", 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7136999999999999e-05 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972656+0000", 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1480000000000001e-05 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140409535000003, 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015429492, 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-13T09:26:22.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957581+0000", 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957580+0000", 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957586+0000", 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1580000000000002e-06 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957620+0000", 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3215e-05 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957651+0000", 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1027e-05 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969570+0000", 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011919511000000001 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969674+0000", 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104248 2024-09-13T09:26:22.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.969784+0000", 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010937700000000001 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972662+0000", 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028781589999999999 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972925+0000", 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026289899999999998 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972976+0000", 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0702999999999999e-05 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973010+0000", 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4743000000000003e-05 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140304931000003, 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020098000000000001, 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.190 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957685+0000", 2024-09-13T09:26:22.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957793+0000", 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108837 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957808+0000", 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.433e-05 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957814+0000", 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3799999999999999e-06 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970251+0000", 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012436523999999999 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970350+0000", 2024-09-13T09:26:22.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9600999999999997e-05 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.970506+0000", 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015539500000000001 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.972667+0000", 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002161619 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977681+0000", 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050135930000000002 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977722+0000", 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1634999999999997e-05 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.977783+0000", 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0968000000000003e-05 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.140168803999998, 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022700502000000001, 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-13T09:26:22.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957822+0000", 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957821+0000", 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957825+0000", 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957843+0000", 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7492000000000001e-05 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.957851+0000", 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4840000000000004e-06 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971666+0000", 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013814707000000001 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971841+0000", 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000174659 2024-09-13T09:26:22.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.971976+0000", 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013485600000000001 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.973161+0000", 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0011852130000000001 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980427+0000", 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072666440000000001 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980487+0000", 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9373000000000002e-05 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.980522+0000", 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5244999999999999e-05 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:4479 3.d 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.262230257000001, 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051456631000000003, 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4479 2024-09-13T09:26:22.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835751+0000", 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999909 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854271+0000", 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01851972 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854305+0000", 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4258000000000003e-05 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854317+0000", 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2013e-05 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854487+0000", 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017018 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854768+0000", 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028067700000000001 2024-09-13T09:26:22.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.855666+0000", 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000898404 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.881583+0000", 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025916947999999999 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887097+0000", 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055134479999999998 2024-09-13T09:26:22.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887161+0000", 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4077000000000003e-05 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887217+0000", 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6128000000000002e-05 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6281 3.13 3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.2509634520000006, 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024029966, 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6281 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847027+0000", 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847025+0000", 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:22.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847238+0000", 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021285799999999999 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847259+0000", 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0908e-05 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847267+0000", 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.67e-06 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847329+0000", 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2016999999999999e-05 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.847464+0000", 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000134497 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.853158+0000", 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056936219999999997 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870613+0000", 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017455404000000001 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870743+0000", 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012992099999999999 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.870911+0000", 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000168276 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:13.871057+0000", 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014575999999999999 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:6591 3.13 3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.9624608439999998, 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015116329, 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 6591 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135530+0000", 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135528+0000", 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135537+0000", 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0669999999999998e-06 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135554+0000", 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135563+0000", 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5380000000000008e-06 2024-09-13T09:26:22.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135634+0000", 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1106999999999995e-05 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.135759+0000", 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012423999999999999 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.139009+0000", 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032509520000000001 2024-09-13T09:26:22.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150356+0000", 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011346962 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150461+0000", 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104998 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150550+0000", 2024-09-13T09:26:22.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8622999999999995e-05 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:14.150646+0000", 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5822000000000005e-05 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:22.219 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:22.221 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:22.217+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:22.221 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:22.322 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:22.321+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:22.339 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:22.340 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:22.340 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:22.340 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:22.350 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:22.414 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:26:22.423 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:22.419+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:22.463 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:22.464 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:22.464 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:22.464 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:22.464 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:22.474 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:22.523 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:22.519+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:22.625 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:22.621+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:22.636 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.681933438999998, 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:22.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:22.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.665375226000002, 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.641 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:22.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.430627968, 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:22.646 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:22.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:22.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:22.652 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.428418830999998, 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:22.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:22.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.424074306000001, 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.657 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:22.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:22.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:22.661 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.407523327, 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.662 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:22.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.406935318999999, 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:22.666 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.668 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:22.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.399229163000001, 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:22.671 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:22.672 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:22.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.675 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.676 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.676 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:22.676 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:22.676 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:22.676 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T09:26:22.677 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:22.726 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:22.723+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:22.800 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:22.800 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:22.800 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:22.800 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:22.812 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:22.827 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:22.823+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:22.924 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:22.924 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:22.924 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:22.924 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:22.924 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:22.929 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:22.925+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:22.934 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:23.029 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:23.029+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:23.092 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4211.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 40.137846091999997, 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051933830000000002, 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4211", 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 1 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948440+0000", 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948437+0000", 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948452+0000", 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4372999999999999e-05 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948471+0000", 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8932000000000001e-05 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948488+0000", 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7002000000000001e-05 2024-09-13T09:26:23.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948542+0000", 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4128000000000001e-05 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.948551+0000", 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1239999999999994e-06 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.951931+0000", 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003380245 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.952258+0000", 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032729100000000001 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.952439+0000", 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018074399999999999 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953396+0000", 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00095699900000000004 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953544+0000", 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014766599999999999 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953591+0000", 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7213999999999999e-05 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:42.953634+0000", 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.099 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:56 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'34, mlcod=18'29)", 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.980740779999998, 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005743209, 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105546+0000", 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105544+0000", 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105557+0000", 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3464e-05 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105607+0000", 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9941999999999997e-05 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105620+0000", 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3049e-05 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105726+0000", 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105185 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.105793+0000", 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7164999999999998e-05 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.111223+0000", 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054298690000000004 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.111289+0000", 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6074999999999993e-05 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.104 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:114 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'76, mlcod=18'71)", 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.707320957, 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005632972, 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378965+0000", 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378964+0000", 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378976+0000", 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1652e-05 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378990+0000", 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.399e-05 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.378999+0000", 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1349999999999998e-06 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379073+0000", 2024-09-13T09:26:23.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4256000000000002e-05 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.379142+0000", 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9139000000000003e-05 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384494+0000", 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053517829999999997 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.384598+0000", 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010432 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:117 2.0 e18/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'79, mlcod=18'72)", 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.704433878000003, 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054346220000000001, 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381852+0000", 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381851+0000", 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381865+0000", 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4103e-05 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381883+0000", 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8522999999999999e-05 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381909+0000", 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5712999999999999e-05 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.381989+0000", 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0086000000000005e-05 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.382020+0000", 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0712e-05 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.387223+0000", 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052033599999999998 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.387287+0000", 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3721999999999997e-05 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:140 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'98, mlcod=18'91)", 2024-09-13T09:26:23.113 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.118115044, 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0068715920000000002, 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968171+0000", 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968170+0000", 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968240+0000", 2024-09-13T09:26:23.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9692999999999998e-05 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968254+0000", 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4268e-05 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968260+0000", 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968307+0000", 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6836999999999999e-05 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.968323+0000", 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6494000000000002e-05 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.974993+0000", 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066696510000000004 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.975043+0000", 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0002e-05 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:141 2.0 e19/17 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'99, mlcod=18'91)", 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.116567580000002, 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080457759999999993, 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.118 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969719+0000", 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969718+0000", 2024-09-13T09:26:23.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969726+0000", 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969754+0000", 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8481000000000001e-05 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969764+0000", 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6509999999999996e-06 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969818+0000", 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4425000000000002e-05 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.969848+0000", 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0122000000000001e-05 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.977718+0000", 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078694349999999993 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.977765+0000", 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6739000000000001e-05 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4166.0:144 2.0 e19/17 2:30306672:devicehealth::main.db.0000000000000000:head v 19'102, mlcod=19'98)", 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.105754539000003, 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058040629999999999, 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980532+0000", 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980531+0000", 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980635+0000", 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010367800000000001 2024-09-13T09:26:23.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980690+0000", 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5795999999999999e-05 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980702+0000", 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1788e-05 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980759+0000", 2024-09-13T09:26:23.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7141000000000001e-05 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.980834+0000", 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4836e-05 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986281+0000", 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054468370000000004 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.986336+0000", 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4115 3.6 3:62bd528f:::rbd_data.1118fb4e889d.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 10747904~4718592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.469589+0000", 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.616697601, 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059536179999999999, 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4115 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469589+0000", 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469589+0000", 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469587+0000", 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469596+0000", 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1970000000000002e-06 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469613+0000", 2024-09-13T09:26:23.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6889999999999999e-05 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469620+0000", 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3490000000000003e-06 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469669+0000", 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8526000000000003e-05 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469784+0000", 2024-09-13T09:26:23.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115207 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.469915+0000", 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000130848 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.471724+0000", 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001808851 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.475404+0000", 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036797729999999999 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.475466+0000", 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2008000000000007e-05 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.475542+0000", 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6637999999999994e-05 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.132 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.132457059, 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:26:23.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:26:23.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.138 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4605 3.19 3:9a4feb2f:::rbd_data.11189fb5ddd6.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.987877+0000", 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.098409776, 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022921418999999998, 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4605 2024-09-13T09:26:23.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987877+0000", 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987877+0000", 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987875+0000", 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987882+0000", 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987895+0000", 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3376e-05 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987902+0000", 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987946+0000", 2024-09-13T09:26:23.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3930999999999997e-05 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.988031+0000", 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5483999999999997e-05 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.991294+0000", 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0032626249999999999 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010644+0000", 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019350046999999999 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010694+0000", 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9910000000000002e-05 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010747+0000", 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010798+0000", 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1146999999999997e-05 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4606 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.011105+0000", 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.075181594, 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062580719999999999, 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4606 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011105+0000", 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011105+0000", 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011103+0000", 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011112+0000", 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8259999999999999e-06 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011124+0000", 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2276000000000001e-05 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011132+0000", 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3050000000000004e-06 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011178+0000", 2024-09-13T09:26:23.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6628999999999997e-05 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.011257+0000", 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9107e-05 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.012137+0000", 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00087923700000000005 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.017272+0000", 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051357110000000003 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.017285+0000", 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2386e-05 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.017317+0000", 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2011000000000002e-05 2024-09-13T09:26:23.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.017363+0000", 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6105e-05 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4614 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [stat,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.037558+0000", 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.048728519000001, 2024-09-13T09:26:23.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007123468, 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4614 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037558+0000", 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037558+0000", 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037557+0000", 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037564+0000", 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000001e-06 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037580+0000", 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6742999999999999e-05 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037590+0000", 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9399999999999997e-06 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037681+0000", 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0711000000000006e-05 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.037762+0000", 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1438000000000002e-05 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.038708+0000", 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00094516700000000001 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044144+0000", 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054361920000000003 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044589+0000", 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044482199999999999 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044643+0000", 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4002999999999998e-05 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.044681+0000", 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8704000000000001e-05 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.157 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4687 3.4 3:26cb438a:::rbd_object_map.11189fb5ddd6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.121853+0000", 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.964433575999999, 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074279940000000003, 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4687 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.121853+0000", 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.121853+0000", 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.121851+0000", 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.121859+0000", 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3930000000000002e-06 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.121870+0000", 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1103e-05 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.121877+0000", 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4460000000000004e-06 2024-09-13T09:26:23.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123932+0000", 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020550490000000002 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123986+0000", 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3767999999999997e-05 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.124200+0000", 2024-09-13T09:26:23.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000213785 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125276+0000", 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001075966 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.129050+0000", 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037743989999999999 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.129173+0000", 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012323100000000001 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.129281+0000", 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107324 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4688 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.125616+0000", 2024-09-13T09:26:23.163 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.960670065, 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0082174640000000007, 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4688 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125616+0000", 2024-09-13T09:26:23.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125616+0000", 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125615+0000", 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125622+0000", 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2639999999999997e-06 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125633+0000", 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1822e-05 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125641+0000", 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5979999999999999e-06 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125837+0000", 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019648499999999999 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.126090+0000", 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025248799999999999 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.126395+0000", 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030552999999999998 2024-09-13T09:26:23.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.129356+0000", 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002961057 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133741+0000", 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043846579999999996 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133795+0000", 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.346e-05 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133834+0000", 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9100999999999997e-05 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4691 3.4 3:26cb438a:::rbd_object_map.11189fb5ddd6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.169 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.127574+0000", 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.95871256, 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063567399999999996, 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4691 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.127574+0000", 2024-09-13T09:26:23.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.127574+0000", 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.127573+0000", 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.127580+0000", 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.204e-06 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.127605+0000", 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4550999999999999e-05 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.127630+0000", 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5582e-05 2024-09-13T09:26:23.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.129459+0000", 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828966 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.129618+0000", 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015872600000000001 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.130712+0000", 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001094009 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133730+0000", 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003018167 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133843+0000", 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112755 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133895+0000", 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2472999999999997e-05 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133930+0000", 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5151999999999998e-05 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.175 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4692 3.4 3:26cb438a:::rbd_object_map.11189fb5ddd6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.128435+0000", 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.957851786999999, 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074251839999999996, 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4692 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.128435+0000", 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.128435+0000", 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.128434+0000", 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.128439+0000", 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3709999999999999e-06 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.128450+0000", 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.112e-05 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.128458+0000", 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6830000000000008e-06 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.131776+0000", 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003318503 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.131844+0000", 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7119000000000006e-05 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132128+0000", 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028457100000000001 2024-09-13T09:26:23.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135311+0000", 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003183047 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135749+0000", 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00043785500000000002 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135819+0000", 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0396999999999997e-05 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135860+0000", 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0269999999999999e-05 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.181 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:21858 3.4 3:27599404:::rbd_data.1118a28cd5c4.0000000000000016:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:17.295401+0000", 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7908854490000001, 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051393569999999998, 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 21858 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295401+0000", 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295401+0000", 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295399+0000", 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295772+0000", 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00037327200000000001 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295793+0000", 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1163e-05 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295804+0000", 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0737000000000001e-05 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.295883+0000", 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8722999999999998e-05 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.296106+0000", 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000222909 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.296606+0000", 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00050023099999999996 2024-09-13T09:26:23.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.299504+0000", 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028979790000000002 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.300419+0000", 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00091485600000000002 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.300492+0000", 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2978999999999994e-05 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:17.300540+0000", 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8347000000000003e-05 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:22892 3.6 3:67089498:::rbd_data.111811db2be7.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.317949+0000", 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.7683370869999999, 2024-09-13T09:26:23.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054160149999999997, 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 22892 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.317949+0000", 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.317949+0000", 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.317946+0000", 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.317961+0000", 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4127e-05 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.317985+0000", 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4501000000000001e-05 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.317996+0000", 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0796e-05 2024-09-13T09:26:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.318100+0000", 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103756 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.318248+0000", 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014794100000000001 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.318388+0000", 2024-09-13T09:26:23.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014055799999999999 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.320157+0000", 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017692420000000001 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.323309+0000", 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031515689999999999 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.323343+0000", 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3840000000000001e-05 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.323365+0000", 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2442999999999998e-05 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23055 3.4 3:209a8a0c:::rbd_data.1118a28cd5c4.0000000000000001:e [read 634880~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.330272+0000", 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.7560143999999998, 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055454670000000001, 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23055 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.330272+0000", 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.330272+0000", 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.330271+0000", 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.330275+0000", 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6679999999999999e-06 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.330284+0000", 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8210000000000008e-06 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.330290+0000", 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5740000000000001e-06 2024-09-13T09:26:23.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335107+0000", 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048162630000000003 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335196+0000", 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9744e-05 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335817+0000", 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00062098999999999998 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:23275 3.18 e44/25 3:1fc8daa1:::rbd_data.111811db2be7.00000000000000d5:head v 44'62, mlcod=44'60)", 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.546258+0000", 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.540028698, 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007254361, 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.198 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546258+0000", 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546258+0000", 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546256+0000", 2024-09-13T09:26:23.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546269+0000", 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2559e-05 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546292+0000", 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3385999999999999e-05 2024-09-13T09:26:23.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546302+0000", 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8679999999999994e-06 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546358+0000", 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5507000000000002e-05 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.546389+0000", 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0919e-05 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.553416+0000", 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070269649999999996 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.553512+0000", 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6461999999999999e-05 2024-09-13T09:26:23.202 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.203 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.203 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.203 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.203 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.203 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:23.204 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1531 read 0x2f80612 thru 0x2f8791c (0x730b bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1532 write 0x8c80dc3 thru 0x8c8cd5f (0xbf9d bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1534 read 0x42dde73 thru 0x42eb157 (0xd2e5 bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1535 read 0x5f1a723 thru 0x5f1c56f (0x1e4d bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1536 punch from 0x3cbb211 to 0x3cc372d, (0x851c bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1537 punch from 0x2f46ac9 to 0x2f561f8, (0xf72f bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1539 write 0x69928af thru 0x699c6b0 (0x9e02 bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1540 writesame 0x291441 thru 0x2a113c data_size 0x3f(0xfcfc bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1541 punch from 0x57a0289 to 0x57a73a9, (0x7120 bytes) 2024-09-13T09:26:23.206 INFO:teuthology.orchestra.run.smithi063.stdout:1542 clone 13 order 20 su 4096 sc 7 2024-09-13T09:26:23.207 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:23.199+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:23.305 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:23.300+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:23.324 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:23.324 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:23.324 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:23.324 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:23.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:23.406 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:23.404+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:23.450 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:23.450 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:23.450 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:23.450 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:23.450 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:23.460 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:23.507 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:23.504+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:23.522 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone12 intact 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:276 3.4 e27/21 3:27927849:::rbd_data.111867511a81.0000000000000026:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.060913630000002, 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002929961, 2024-09-13T09:26:23.571 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504324+0000", 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504366+0000", 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2412e-05 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504407+0000", 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0329e-05 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504427+0000", 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0089999999999999e-05 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504518+0000", 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0801e-05 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504566+0000", 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8745999999999997e-05 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507192+0000", 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00262546 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507256+0000", 2024-09-13T09:26:23.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:307 3.9 e27/21 3:92c4c1be:::rbd_data.111867511a81.0000000000000087:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.935667158000001, 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002303486, 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.576 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629570+0000", 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629585+0000", 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4826e-05 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629615+0000", 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9879e-05 2024-09-13T09:26:23.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629628+0000", 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2673e-05 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629749+0000", 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121304 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629816+0000", 2024-09-13T09:26:23.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7285000000000004e-05 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.631763+0000", 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019471040000000001 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.631876+0000", 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112151 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.580 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.762300634999999, 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:26:23.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:26:23.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.35412475, 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.585 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:26:23.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.353934627000001, 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:26:23.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:26:23.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.353122217999999, 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2983 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'24, mlcod=31'22)", 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.243457+0000", 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.321782666000001, 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020652140000000001, 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243457+0000", 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243457+0000", 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243455+0000", 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243592+0000", 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013771700000000001 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243624+0000", 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1983999999999997e-05 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243639+0000", 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.518e-05 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243738+0000", 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8302999999999997e-05 2024-09-13T09:26:23.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.243784+0000", 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6385999999999999e-05 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.245394+0000", 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001609542 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.245522+0000", 2024-09-13T09:26:23.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012801699999999999 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.164215569, 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:26:23.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3708 3.9 e34/21 3:92235de7:::rbd_data.111851704b2b.0000000000000027:head v 34'42, mlcod=32'39)", 2024-09-13T09:26:23.607 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:08.438034+0000", 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.127205529999999, 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021117089999999998, 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438034+0000", 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438034+0000", 2024-09-13T09:26:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438033+0000", 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438043+0000", 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3899999999999999e-06 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438069+0000", 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438087+0000", 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7286e-05 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438197+0000", 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109978 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.438235+0000", 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8621999999999997e-05 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.440048+0000", 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018124269999999999 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:08.440145+0000", 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7733999999999996e-05 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.611 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4109 3.1f e35/25 3:fb0b19d8:::rbd_data.1118fb4e889d.0000000000000006:head v 35'26, mlcod=31'23)", 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.099144798999999, 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028372279999999998, 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:23.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466099+0000", 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7079999999999996e-06 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466120+0000", 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1491000000000001e-05 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466130+0000", 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466228+0000", 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8072000000000006e-05 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466258+0000", 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9842e-05 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.468885+0000", 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026277119999999999 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.468932+0000", 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6322000000000001e-05 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4374 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'95, mlcod=35'93)", 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.764134+0000", 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.801104948000001, 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002081795, 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.616 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764134+0000", 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764134+0000", 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764134+0000", 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764141+0000", 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1269999999999997e-06 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764152+0000", 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1153e-05 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764159+0000", 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1740000000000003e-06 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764217+0000", 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7751999999999999e-05 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.764246+0000", 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8801000000000001e-05 2024-09-13T09:26:23.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.766182+0000", 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019360639999999999 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.766216+0000", 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4322000000000001e-05 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.599106184, 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:26:23.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:26:23.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.525567171, 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.625 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:26:23.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.629 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4685 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'114, mlcod=35'111)", 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.123232+0000", 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.442006897000001, 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00221652, 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123232+0000", 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123232+0000", 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123232+0000", 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123243+0000", 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1905999999999999e-05 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123267+0000", 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3989999999999999e-05 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123288+0000", 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1053e-05 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123754+0000", 2024-09-13T09:26:23.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00046587400000000002 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.123772+0000", 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7309999999999999e-05 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125415+0000", 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001643802 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.125449+0000", 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3361999999999997e-05 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.433023802999999, 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:23.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4694 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'120, mlcod=35'115)", 2024-09-13T09:26:23.638 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.133308+0000", 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.431930842, 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002153786, 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133308+0000", 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133308+0000", 2024-09-13T09:26:23.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133307+0000", 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133315+0000", 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8860000000000004e-06 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133331+0000", 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5545000000000001e-05 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133339+0000", 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9959999999999993e-06 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133375+0000", 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6604999999999999e-05 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.133398+0000", 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2690000000000001e-05 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135434+0000", 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020357090000000001 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135462+0000", 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8246999999999999e-05 2024-09-13T09:26:23.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:6417 3.1f e38/25 3:f9e12ce1:::rbd_data.1118e98b86fc.0000000000000024:head v 38'28, mlcod=35'26)", 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:13.992398+0000", 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.5728408690000002, 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021331290000000001, 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992398+0000", 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992398+0000", 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992396+0000", 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992485+0000", 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8294999999999994e-05 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992519+0000", 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3896000000000002e-05 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992533+0000", 2024-09-13T09:26:23.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4857e-05 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992633+0000", 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9646000000000001e-05 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.992678+0000", 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4666000000000002e-05 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.994410+0000", 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001732585 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:13.994531+0000", 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121174 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:6576 3.1f e38/25 3:fff74333:::rbd_data.1118e98b86fc.0000000000000003:head v 38'31, mlcod=38'29)", 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:14.090448+0000", 2024-09-13T09:26:23.647 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.4747915190000001, 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002258323, 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090448+0000", 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090448+0000", 2024-09-13T09:26:23.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090446+0000", 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090812+0000", 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036630899999999999 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090849+0000", 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7054000000000001e-05 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090863+0000", 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3583e-05 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090952+0000", 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9294000000000003e-05 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090982+0000", 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9663e-05 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.092630+0000", 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016485440000000001 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.092706+0000", 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5869000000000002e-05 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:6645 3.0 e38/21 3:017dfdfc:::rbd_data.1118e98b86fc.0000000000000000:head v 38'83, mlcod=37'81)", 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:14.204438+0000", 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3608013260000007, 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022137889999999999, 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.652 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204438+0000", 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204438+0000", 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204435+0000", 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204661+0000", 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000225164 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204693+0000", 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2462000000000002e-05 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204714+0000", 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0539000000000001e-05 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204810+0000", 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6255999999999995e-05 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.204860+0000", 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9914999999999999e-05 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.206587+0000", 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017274 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.206652+0000", 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4444999999999997e-05 2024-09-13T09:26:23.656 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:24464 3.9 e45/21 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:head v 45'68, mlcod=44'66)", 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.41342807599999998, 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028365740000000001, 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.657 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151810+0000", 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:23.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152495+0000", 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00068434500000000005 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152521+0000", 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6288e-05 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:23.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152534+0000", 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152635+0000", 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100758 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152669+0000", 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4826999999999997e-05 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.154591+0000", 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001921407 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.154648+0000", 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6826000000000001e-05 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:23.661 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:23.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:23.663 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:23.659+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:23.680 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T09:26:23.764 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:23.760+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:23.788 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:23.788 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:23.788 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:23.788 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:23.799 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:26:23.863 INFO:teuthology.orchestra.run.smithi063.stdout:1544 punch from 0x4bc3fad to 0x4bc77df, (0x3832 bytes) 2024-09-13T09:26:23.864 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:23.860+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:23.882 INFO:teuthology.orchestra.run.smithi063.stdout:1545 writesame 0xa53fad1 thru 0xa54051f data_size 0x1(0xa4f bytes) 2024-09-13T09:26:23.882 INFO:teuthology.orchestra.run.smithi063.stdout:1546 write 0x5b4c7ec thru 0x5b51329 (0x4b3e bytes) 2024-09-13T09:26:23.909 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:23.910 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:23.910 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:23.910 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:23.910 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:23.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:26:23.934 INFO:teuthology.orchestra.run.smithi063.stdout:1547 trunc from 0xa540520 to 0x412f138 2024-09-13T09:26:23.964 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:23.959+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:24.012 INFO:teuthology.orchestra.run.smithi063.stdout:1549 trunc from 0x412f138 to 0x6471b61 2024-09-13T09:26:24.023 INFO:teuthology.orchestra.run.smithi063.stdout:1552 read 0x40c7fbd thru 0x40cc833 (0x4877 bytes) 2024-09-13T09:26:24.041 INFO:teuthology.orchestra.run.smithi063.stdout:1553 write 0x8f85fac thru 0x8f910e4 (0xb139 bytes) 2024-09-13T09:26:24.048 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:448 3.8 3:16185ba7:::rbd_data.111867511a81.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.090511601999999, 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022114892000000001, 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 448 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952827+0000", 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952824+0000", 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952840+0000", 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5420000000000001e-05 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952875+0000", 2024-09-13T09:26:24.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5076000000000001e-05 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.952890+0000", 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5596000000000001e-05 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.953033+0000", 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014238400000000001 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.953282+0000", 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024892700000000002 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.959632+0000", 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.006350372 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974849+0000", 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015217229000000001 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:24.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974877+0000", 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7421999999999999e-05 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974914+0000", 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7320000000000002e-05 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.974942+0000", 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7763000000000002e-05 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2265 3.2 3:43c4f885:::rbd_data.1118e3e5e25d.000000000000019f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.553687020000002, 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017493669999999999, 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2265 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.055 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489651+0000", 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489650+0000", 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:24.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489658+0000", 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4939999999999997e-06 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489675+0000", 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7384000000000002e-05 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489683+0000", 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.038e-06 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489735+0000", 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1824999999999999e-05 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.489826+0000", 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1533999999999995e-05 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:24.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.491305+0000", 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014785320000000001 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.501618+0000", 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010312928000000001 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.506959+0000", 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053409310000000002 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:24.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.507065+0000", 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105925 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.507145+0000", 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0455000000000001e-05 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:2815 3.1e 3:7b894b3d:::rbd_data.11189888a71f.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.159701971, 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02585463, 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 2815 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883636+0000", 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883635+0000", 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883648+0000", 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3205e-05 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883682+0000", 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4421000000000001e-05 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883700+0000", 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7501000000000001e-05 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883780+0000", 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0418000000000002e-05 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.883885+0000", 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105368 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.889905+0000", 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0060196479999999998 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909412+0000", 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019506672999999999 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909424+0000", 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2692e-05 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909462+0000", 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7069e-05 2024-09-13T09:26:24.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:04.909491+0000", 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9560000000000002e-05 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4565 3.b 3:d5801e6a:::rbd_data.11189fb5ddd6.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10863232~40683] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.904991+0000", 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.138347269, 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.038272699, 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:24.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4565 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.904991+0000", 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.904991+0000", 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.904989+0000", 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913214+0000", 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0082248500000000006 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913250+0000", 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6724999999999998e-05 2024-09-13T09:26:24.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913263+0000", 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2506e-05 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913351+0000", 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8676000000000003e-05 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.913597+0000", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024513900000000001 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.915014+0000", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001417062 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.935717+0000", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020703565 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.943108+0000", 2024-09-13T09:26:24.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0073912769999999999 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.943178+0000", 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9757000000000003e-05 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.943264+0000", 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5612000000000006e-05 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:22454 3.2 3:4749434a:::rbd_data.111811db2be7.00000000000000c1:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:18.901764+0000", 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.1415741510000004, 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035243253000000002, 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 22454 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901764+0000", 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901764+0000", 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901762+0000", 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901776+0000", 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4243999999999999e-05 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901809+0000", 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3164000000000003e-05 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901824+0000", 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4677e-05 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.901931+0000", 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107149 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.902280+0000", 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00034854999999999999 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.910783+0000", 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085038409999999998 2024-09-13T09:26:24.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.936847+0000", 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.026063123000000001 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.936871+0000", 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4008000000000001e-05 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.936949+0000", 2024-09-13T09:26:24.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8307999999999996e-05 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.937008+0000", 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8600999999999997e-05 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:22506 3.18 3:195db564:::rbd_data.111811db2be7.00000000000000d3:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:18.966781+0000", 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0765576689999996, 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.021175487, 2024-09-13T09:26:24.077 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 22506 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966781+0000", 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966781+0000", 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966779+0000", 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966790+0000", 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0956e-05 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966810+0000", 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0205999999999999e-05 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966821+0000", 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0644e-05 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.966869+0000", 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8099999999999997e-05 2024-09-13T09:26:24.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.967009+0000", 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000140183 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.970006+0000", 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029970259999999999 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:24.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.987819+0000", 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017813269 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.987838+0000", 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8546000000000002e-05 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.987907+0000", 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8800999999999994e-05 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:18.987956+0000", 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9642999999999999e-05 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23146 3.8 3:1752c46c:::rbd_data.1118a28cd5c4.0000000000000000:e [read 503808~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335387+0000", 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7079514109999998, 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015269213, 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23146 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335387+0000", 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335387+0000", 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335387+0000", 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335390+0000", 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3289999999999998e-06 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335397+0000", 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2529999999999998e-06 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335403+0000", 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6670000000000001e-06 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350472+0000", 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015068787 2024-09-13T09:26:24.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350553+0000", 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1071999999999999e-05 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350656+0000", 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103565 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23147 3.8 3:1752c46c:::rbd_data.1118a28cd5c4.0000000000000000:e [read 679936~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335408+0000", 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7079305399999996, 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016312970999999999, 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23147 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335408+0000", 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335408+0000", 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335407+0000", 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335420+0000", 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2608e-05 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335428+0000", 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.926e-06 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335434+0000", 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7459999999999997e-06 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351536+0000", 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016101969000000001 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351622+0000", 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6403999999999995e-05 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351721+0000", 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8774e-05 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23160 3.18 3:1c7920ee:::rbd_data.1118a28cd5c4.0000000000000004:e [read 151552~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335439+0000", 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.707899072, 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015417808, 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23160 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335439+0000", 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335439+0000", 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335439+0000", 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335447+0000", 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0870000000000007e-06 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335455+0000", 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7950000000000008e-06 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335461+0000", 2024-09-13T09:26:24.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350679+0000", 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01521845 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350757+0000", 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8119000000000002e-05 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350857+0000", 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9949000000000001e-05 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23161 3.18 3:1c7920ee:::rbd_data.1118a28cd5c4.0000000000000004:e [read 327680~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335467+0000", 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7078717650000002, 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015598045, 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23161 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335467+0000", 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335467+0000", 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335466+0000", 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335469+0000", 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2499999999999998e-06 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335515+0000", 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5463000000000002e-05 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335523+0000", 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6799999999999993e-06 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350883+0000", 2024-09-13T09:26:24.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015360189 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.350965+0000", 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2411999999999996e-05 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351065+0000", 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9531000000000006e-05 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23162 3.18 3:1c7920ee:::rbd_data.1118a28cd5c4.0000000000000004:e [read 503808~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335530+0000", 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7078083920000005, 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015747071000000001, 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23162 2024-09-13T09:26:24.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335530+0000", 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335530+0000", 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335529+0000", 2024-09-13T09:26:24.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335533+0000", 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.138e-06 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335542+0000", 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4889999999999995e-06 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335548+0000", 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1229999999999997e-06 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351089+0000", 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01554116 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351173+0000", 2024-09-13T09:26:24.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3534000000000004e-05 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351277+0000", 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010446300000000001 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23163 3.18 3:1c7920ee:::rbd_data.1118a28cd5c4.0000000000000004:e [read 675840~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335553+0000", 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7077851810000002, 2024-09-13T09:26:24.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015955558000000002, 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23163 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335553+0000", 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335553+0000", 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335553+0000", 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335556+0000", 2024-09-13T09:26:24.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335564+0000", 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9270000000000005e-06 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335577+0000", 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3772e-05 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351301+0000", 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015723892999999999 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351384+0000", 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2936999999999996e-05 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351509+0000", 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012445699999999999 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23172 3.8 3:121fe3f4:::rbd_data.1118a28cd5c4.0000000000000007:e [read 151552~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335608+0000", 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7077302000000003, 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016327688, 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23172 2024-09-13T09:26:24.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335608+0000", 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335608+0000", 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335608+0000", 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335612+0000", 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1370000000000004e-06 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335621+0000", 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8300000000000002e-06 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335626+0000", 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9320000000000001e-06 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351755+0000", 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016128139999999999 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351837+0000", 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2825999999999996e-05 2024-09-13T09:26:24.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351936+0000", 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8448000000000003e-05 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23173 3.8 3:121fe3f4:::rbd_data.1118a28cd5c4.0000000000000007:e [read 327680~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335632+0000", 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7077066720000005, 2024-09-13T09:26:24.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016519025999999999, 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23173 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335632+0000", 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335632+0000", 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335631+0000", 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335635+0000", 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4659999999999999e-06 2024-09-13T09:26:24.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335642+0000", 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6170000000000001e-06 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335648+0000", 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.351967+0000", 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016319196000000001 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352050+0000", 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3086999999999999e-05 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352151+0000", 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100462 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23174 3.8 3:121fe3f4:::rbd_data.1118a28cd5c4.0000000000000007:e [read 503808~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335653+0000", 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7076850019999998, 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016708147, 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23174 2024-09-13T09:26:24.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335653+0000", 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335653+0000", 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335653+0000", 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335656+0000", 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335663+0000", 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4259999999999997e-06 2024-09-13T09:26:24.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335669+0000", 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3929999999999999e-06 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352179+0000", 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016510147999999999 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352261+0000", 2024-09-13T09:26:24.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2213999999999996e-05 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352362+0000", 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010056300000000001 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23175 3.8 3:121fe3f4:::rbd_data.1118a28cd5c4.0000000000000007:e [read 675840~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335674+0000", 2024-09-13T09:26:24.124 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7076646499999999, 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016903524999999999, 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23175 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335674+0000", 2024-09-13T09:26:24.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335674+0000", 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335673+0000", 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335676+0000", 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9490000000000001e-06 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335684+0000", 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7989999999999994e-06 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335690+0000", 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4410000000000001e-06 2024-09-13T09:26:24.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352394+0000", 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016704192999999999 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352478+0000", 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3941000000000002e-05 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352577+0000", 2024-09-13T09:26:24.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9633e-05 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23180 3.8 3:1269e70a:::rbd_data.1118a28cd5c4.0000000000000009:e [read 151552~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335695+0000", 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7076435869999997, 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017085855, 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23180 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335695+0000", 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335695+0000", 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335694+0000", 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335697+0000", 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9050000000000001e-06 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335704+0000", 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1230000000000002e-06 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335711+0000", 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.161e-06 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352607+0000", 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016896193 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352685+0000", 2024-09-13T09:26:24.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8387999999999996e-05 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352781+0000", 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5554e-05 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23181 3.8 3:1269e70a:::rbd_data.1118a28cd5c4.0000000000000009:e [read 327680~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335787+0000", 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7075514920000003, 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017208976000000001, 2024-09-13T09:26:24.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23181 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335787+0000", 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335787+0000", 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335786+0000", 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335795+0000", 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335806+0000", 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0736e-05 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335813+0000", 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3509999999999996e-06 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352808+0000", 2024-09-13T09:26:24.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016994893 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352893+0000", 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4361000000000001e-05 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.352996+0000", 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010331300000000001 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23182 3.8 3:1269e70a:::rbd_data.1118a28cd5c4.0000000000000009:e [read 499712~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335820+0000", 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7075185150000003, 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017387149000000001, 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23182 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335820+0000", 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335820+0000", 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335819+0000", 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335823+0000", 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6729999999999998e-06 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335831+0000", 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335837+0000", 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7910000000000001e-06 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.353026+0000", 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017188528000000002 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.353106+0000", 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0332000000000003e-05 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.353207+0000", 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010091599999999999 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:23183 3.8 3:1269e70a:::rbd_data.1118a28cd5c4.0000000000000009:e [read 675840~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:19.335843+0000", 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.7074955059999999, 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017565315000000001, 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:24.142 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 23183 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335843+0000", 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335843+0000", 2024-09-13T09:26:24.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335842+0000", 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335846+0000", 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1540000000000002e-06 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:24.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335853+0000", 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0380000000000002e-06 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.335856+0000", 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4170000000000001e-06 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.353235+0000", 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.017379202999999999 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:24.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.353318+0000", 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2677999999999997e-05 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:19.353408+0000", 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0364000000000004e-05 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:24.146 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:24.147 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:24.148 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:24.144+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:24.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1556 write 0xc3b73fe thru 0xc3bd70c (0x630f bytes) 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1558 read 0x6cc8702 thru 0x6cd69cf (0xe2ce bytes) 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1559 write 0x77ad7f2 thru 0x77bbe32 (0xe641 bytes) 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1561 punch from 0x247b315 to 0x247c604, (0x12ef bytes) 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1562 trunc from 0xc3bd70d to 0xd9f4ccf 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1563 punch from 0xb702593 to 0xb70fd4a, (0xd7b7 bytes) 2024-09-13T09:26:24.150 INFO:teuthology.orchestra.run.smithi063.stdout:1564 write 0x121ed2c thru 0x12291a9 (0xa47e bytes) 2024-09-13T09:26:24.186 INFO:teuthology.orchestra.run.smithi063.stdout:1566 read 0x3527be4 thru 0x35379ca (0xfde7 bytes) 2024-09-13T09:26:24.196 INFO:teuthology.orchestra.run.smithi063.stdout:1567 trunc from 0xd9f4ccf to 0xe9b2339 2024-09-13T09:26:24.234 INFO:teuthology.orchestra.run.smithi063.stdout:1568 write 0x2e7402e thru 0x2e821fb (0xe1ce bytes) 2024-09-13T09:26:24.248 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:24.243+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:24.279 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.280 INFO:teuthology.orchestra.run.smithi063.stdout:1570 writesame 0x8a76f2e thru 0x8a7c25a data_size 0x1(0x532d bytes) 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:27465 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.279702+0000", 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054236639999999999, 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005447637, 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 27465 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279702+0000", 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279702+0000", 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279702+0000", 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279704+0000", 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0339999999999999e-06 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279712+0000", 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5120000000000008e-06 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279715+0000", 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1300000000000001e-06 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.281967+0000", 2024-09-13T09:26:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022515109999999999 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.282043+0000", 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6332e-05 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.282347+0000", 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030433800000000003 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:27466 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.279719+0000", 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054063369999999998, 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055030110000000004, 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 27466 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279719+0000", 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279719+0000", 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279719+0000", 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279721+0000", 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9690000000000001e-06 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279724+0000", 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3610000000000001e-06 2024-09-13T09:26:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279727+0000", 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7800000000000001e-06 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283111+0000", 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003384176 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283175+0000", 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3900999999999997e-05 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283464+0000", 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028830799999999998 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:27467 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.279743+0000", 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0053825949999999996, 2024-09-13T09:26:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055376460000000002, 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 27467 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279743+0000", 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279743+0000", 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279743+0000", 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279745+0000", 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5859999999999999e-06 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279749+0000", 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1030000000000004e-06 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279753+0000", 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.191e-06 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283809+0000", 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040569040000000001 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283867+0000", 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7596000000000002e-05 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284133+0000", 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026637299999999998 2024-09-13T09:26:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:27468 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.279766+0000", 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0053595700000000001, 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055866400000000004, 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 27468 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279766+0000", 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279766+0000", 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279766+0000", 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279768+0000", 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.553e-06 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279772+0000", 2024-09-13T09:26:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2069999999999997e-06 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.279775+0000", 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284587+0000", 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0048115279999999998 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284674+0000", 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6836999999999995e-05 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.285112+0000", 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00043867100000000002 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:27471 3.3 3:c09cf4cd:::rbd_data.111812ca204d.0000000000000087:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.283204+0000", 2024-09-13T09:26:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001921212, 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022183770000000001, 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 27471 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283204+0000", 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283204+0000", 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283203+0000", 2024-09-13T09:26:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283210+0000", 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9659999999999999e-06 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283220+0000", 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8549999999999997e-06 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283227+0000", 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283301+0000", 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4542000000000006e-05 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283413+0000", 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011144700000000001 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.283903+0000", 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00049055799999999999 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:27465 3.1 e46/25)", 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.284085+0000", 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001040556, 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001383041, 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284085+0000", 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284085+0000", 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284084+0000", 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284090+0000", 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4650000000000002e-06 2024-09-13T09:26:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284100+0000", 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284106+0000", 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7190000000000004e-06 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4376.0:27466 3.1 e46/25)", 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.284573+0000", 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00055219899999999996, 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00091613700000000003, 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284573+0000", 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284573+0000", 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284573+0000", 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284579+0000", 2024-09-13T09:26:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1670000000000004e-06 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284588+0000", 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3549999999999994e-06 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.284596+0000", 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9370000000000004e-06 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:26:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 7 2024-09-13T09:26:24.316 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:24.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1571 read 0x5afb157 thru 0x5b02dc6 (0x7c70 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1572 read 0xa4f135d thru 0xa4f6655 (0x52f9 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1574 read 0x72da655 thru 0x72e37c8 (0x9174 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1575 read 0xb2e7337 thru 0xb2ec109 (0x4dd3 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1576 write 0x4913ec5 thru 0x4915dca (0x1f06 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1578 read 0xcd2872e thru 0xcd35232 (0xcb05 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1579 trunc from 0xe9b2339 to 0xc02a6fd 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1581 write 0xe4c8c2 thru 0xe51e17 (0x5556 bytes) 2024-09-13T09:26:24.319 INFO:teuthology.orchestra.run.smithi063.stdout:1582 write 0x90c742 thru 0x90f3fb (0x2cba bytes) 2024-09-13T09:26:24.340 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:24.349 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:24.344+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:24.351 INFO:teuthology.orchestra.run.smithi063.stdout:1583 write 0xa3ce8ab thru 0xa3d2eff (0x4655 bytes) 2024-09-13T09:26:24.361 INFO:teuthology.orchestra.run.smithi063.stdout:1584 punch from 0x36474cc to 0x364abc9, (0x36fd bytes) 2024-09-13T09:26:24.361 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.361 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.362 INFO:teuthology.orchestra.run.smithi063.stdout:1590 write 0xb0bf3a5 thru 0xb0c5e53 (0x6aaf bytes) 2024-09-13T09:26:24.364 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:24.377 INFO:teuthology.orchestra.run.smithi063.stdout:1592 read 0xaebaadb thru 0xaec207f (0x75a5 bytes) 2024-09-13T09:26:24.388 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:24.400 INFO:teuthology.orchestra.run.smithi063.stdout:1594 trunc from 0xc02a6fd to 0x21f843 2024-09-13T09:26:24.418 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:26:24.428 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:26:24.446 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:24.446 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:24.446 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:24.447 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:24.447 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:24.449 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:24.453 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:24.448+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:24.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:24.483 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:24.511 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:24.531 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:24.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:24.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:24.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:24.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:26:24.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:24.534 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:24.534 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:24.534 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:24.551 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:24.548+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:24.558 INFO:teuthology.orchestra.run.smithi063.stdout:1595 writesame 0x4927920 thru 0x4928689 data_size 0x11(0xd6a bytes) 2024-09-13T09:26:24.572 INFO:teuthology.orchestra.run.smithi063.stdout:1596 writesame 0x81956d7 thru 0x81a041c data_size 0x3(0xad46 bytes) 2024-09-13T09:26:24.572 INFO:teuthology.orchestra.run.smithi063.stdout:1599 read 0x668598d thru 0x668a025 (0x4699 bytes) 2024-09-13T09:26:24.572 INFO:teuthology.orchestra.run.smithi063.stdout:1600 read 0x3fb4111 thru 0x3fb43b9 (0x2a9 bytes) 2024-09-13T09:26:24.583 INFO:teuthology.orchestra.run.smithi063.stdout:1601 writesame 0x9ae40a8 thru 0x9ae5950 data_size 0x1(0x18a9 bytes) 2024-09-13T09:26:24.598 INFO:teuthology.orchestra.run.smithi063.stdout:1602 write 0xd6e825b thru 0xd6ebd73 (0x3b19 bytes) 2024-09-13T09:26:24.618 INFO:teuthology.orchestra.run.smithi063.stdout:1603 trunc from 0xd6ebd74 to 0x901a8a3 2024-09-13T09:26:24.626 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:24.626 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:24.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:24.636 INFO:teuthology.orchestra.run.smithi063.stdout:1604 write 0x5c2e9ac thru 0x5c35b24 (0x7179 bytes) 2024-09-13T09:26:24.662 INFO:teuthology.orchestra.run.smithi063.stdout:1605 trunc from 0x901a8a3 to 0x2ea8ed5 2024-09-13T09:26:24.681 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:24.680+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:24.700 INFO:teuthology.orchestra.run.smithi063.stdout:1606 writesame 0xa126230 thru 0xa12b936 data_size 0x1(0x5707 bytes) 2024-09-13T09:26:24.700 INFO:teuthology.orchestra.run.smithi063.stdout:1608 write 0x4e6b118 thru 0x4e74e65 (0x9d4e bytes) 2024-09-13T09:26:24.717 INFO:teuthology.orchestra.run.smithi063.stdout:1609 write 0x579d01c thru 0x57ac54b (0xf530 bytes) 2024-09-13T09:26:24.729 INFO:teuthology.orchestra.run.smithi063.stdout:1610 read 0x7b11764 thru 0x7b14ebd (0x375a bytes) 2024-09-13T09:26:24.729 INFO:teuthology.orchestra.run.smithi063.stdout:1611 read 0x6ac7953 thru 0x6acb827 (0x3ed5 bytes) 2024-09-13T09:26:24.732 INFO:teuthology.orchestra.run.smithi063.stdout:1613 trunc from 0xa12b937 to 0xbe092c2 2024-09-13T09:26:24.746 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:24.746 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:24.749 INFO:teuthology.orchestra.run.smithi063.stdout:1614 write 0xbfd1c13 thru 0xbfe0c18 (0xf006 bytes) 2024-09-13T09:26:24.752 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:24.747+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:24.756 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:24.767 INFO:teuthology.orchestra.run.smithi063.stdout:1616 write 0xa47a818 thru 0xa48653f (0xbd28 bytes) 2024-09-13T09:26:24.784 INFO:teuthology.orchestra.run.smithi063.stdout:1617 write 0x1453645 thru 0x1456939 (0x32f5 bytes) 2024-09-13T09:26:24.793 INFO:teuthology.orchestra.run.smithi063.stdout:1619 write 0x52b6b79 thru 0x52c3ed6 (0xd35e bytes) 2024-09-13T09:26:24.807 INFO:teuthology.orchestra.run.smithi063.stdout:1620 write 0xc14d3ce thru 0xc152ce0 (0x5913 bytes) 2024-09-13T09:26:24.812 INFO:teuthology.orchestra.run.smithi063.stdout:1621 punch from 0x67d4698 to 0x67d856a, (0x3ed2 bytes) 2024-09-13T09:26:24.812 INFO:teuthology.orchestra.run.smithi063.stdout:1624 punch from 0x77600dd to 0x77635bf, (0x34e2 bytes) 2024-09-13T09:26:24.812 INFO:teuthology.orchestra.run.smithi063.stdout:1625 read 0xa7c5524 thru 0xa7d2e27 (0xd904 bytes) 2024-09-13T09:26:24.823 INFO:teuthology.orchestra.run.smithi063.stdout:1630 write 0xd7568f2 thru 0xd75c423 (0x5b32 bytes) 2024-09-13T09:26:24.840 INFO:teuthology.orchestra.run.smithi063.stdout:1631 punch from 0x418111c to 0x418cfc3, (0xbea7 bytes) 2024-09-13T09:26:24.840 INFO:teuthology.orchestra.run.smithi063.stdout:1632 read 0x570309e thru 0x570b741 (0x86a4 bytes) 2024-09-13T09:26:24.841 INFO:teuthology.orchestra.run.smithi063.stdout:1634 write 0x258d103 thru 0x2597f36 (0xae34 bytes) 2024-09-13T09:26:24.853 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:24.847+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:24.854 INFO:teuthology.orchestra.run.smithi063.stdout:1635 writesame 0x5e1a57 thru 0x5f0b84 data_size 0x1(0xf12e bytes) 2024-09-13T09:26:24.867 INFO:teuthology.orchestra.run.smithi063.stdout:1636 write 0xa9f8177 thru 0xa9f8389 (0x213 bytes) 2024-09-13T09:26:24.871 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.871 INFO:teuthology.orchestra.run.smithi063.stdout:1639 writesame 0x33ca539 thru 0x33d2241 data_size 0x1(0x7d09 bytes) 2024-09-13T09:26:24.873 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:24.873 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:24.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:24.889 INFO:teuthology.orchestra.run.smithi063.stdout:1640 write 0x1feb861 thru 0x1fedeff (0x269f bytes) 2024-09-13T09:26:24.895 INFO:teuthology.orchestra.run.smithi063.stdout:1641 writesame 0xbfbfab2 thru 0xbfc94fa data_size 0x1(0x9a49 bytes) 2024-09-13T09:26:24.904 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.905 INFO:teuthology.orchestra.run.smithi063.stdout:1645 write 0xa0103c1 thru 0xa014066 (0x3ca6 bytes) 2024-09-13T09:26:24.928 INFO:teuthology.orchestra.run.smithi063.stdout:1646 trunc from 0xd75c424 to 0x82e57d4 2024-09-13T09:26:24.948 INFO:teuthology.orchestra.run.smithi063.stdout:1647 read 0x2a87850 thru 0x2a8c759 (0x4f0a bytes) 2024-09-13T09:26:24.949 INFO:teuthology.orchestra.run.smithi063.stdout:1648 punch from 0x693b27 to 0x69fda9, (0xc282 bytes) 2024-09-13T09:26:24.949 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.949 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:24.949 INFO:teuthology.orchestra.run.smithi063.stdout:1653 writesame 0x34562f thru 0x349b48 data_size 0x3d(0x451a bytes) 2024-09-13T09:26:24.953 INFO:teuthology.orchestra.run.smithi063.stdout:1654 read 0x7304536 thru 0x730892e (0x43f9 bytes) 2024-09-13T09:26:24.954 INFO:teuthology.orchestra.run.smithi063.stdout:1655 read 0x2b4e4b6 thru 0x2b50d68 (0x28b3 bytes) 2024-09-13T09:26:24.954 INFO:teuthology.orchestra.run.smithi063.stdout:1656 write 0x40bcc93 thru 0x40c5848 (0x8bb6 bytes) 2024-09-13T09:26:24.954 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:24.951+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:24.971 INFO:teuthology.orchestra.run.smithi063.stdout:1658 trunc from 0x82e57d4 to 0x9296ffe 2024-09-13T09:26:24.983 INFO:teuthology.orchestra.run.smithi063.stdout:1660 writesame 0x497822c thru 0x49825e2 data_size 0x1(0xa3b7 bytes) 2024-09-13T09:26:24.995 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:24.996 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.004 INFO:teuthology.orchestra.run.smithi063.stdout:1661 write 0x2987404 thru 0x2995eb8 (0xeab5 bytes) 2024-09-13T09:26:25.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:25.010 INFO:teuthology.orchestra.run.smithi063.stdout:1662 trunc from 0x9296ffe to 0xa54bdef 2024-09-13T09:26:25.019 INFO:teuthology.orchestra.run.smithi063.stdout:1663 read 0x425f9e9 thru 0x4263b41 (0x4159 bytes) 2024-09-13T09:26:25.037 INFO:teuthology.orchestra.run.smithi063.stdout:1664 writesame 0xe5b8255 thru 0xe5c67de data_size 0xb(0xe58a bytes) 2024-09-13T09:26:25.037 INFO:teuthology.orchestra.run.smithi063.stdout:1665 read 0x4395ad0 thru 0x43a548f (0xf9c0 bytes) 2024-09-13T09:26:25.038 INFO:teuthology.orchestra.run.smithi063.stdout:1666 write 0x269cce6 thru 0x26a80a3 (0xb3be bytes) 2024-09-13T09:26:25.043 INFO:teuthology.orchestra.run.smithi063.stdout:1667 flatten 2024-09-13T09:26:25.054 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:25.052+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:25.055 INFO:teuthology.orchestra.run.smithi063.stdout:1668 read 0x910494d thru 0x9111b49 (0xd1fd bytes) 2024-09-13T09:26:25.055 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:25.055 INFO:teuthology.orchestra.run.smithi063.stdout:1670 write 0xb494181 thru 0xb49edfd (0xac7d bytes) 2024-09-13T09:26:25.073 INFO:teuthology.orchestra.run.smithi063.stdout:1671 punch from 0x3d7511e to 0x3d842df, (0xf1c1 bytes) 2024-09-13T09:26:25.073 INFO:teuthology.orchestra.run.smithi063.stdout:1672 read 0x8bfb6b thru 0x8cf529 (0xf9bf bytes) 2024-09-13T09:26:25.073 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:25.073 INFO:teuthology.orchestra.run.smithi063.stdout:1674 write 0xa690c6e thru 0xa696071 (0x5404 bytes) 2024-09-13T09:26:25.087 INFO:teuthology.orchestra.run.smithi063.stdout:1675 write 0x8615446 thru 0x8622c38 (0xd7f3 bytes) 2024-09-13T09:26:25.100 INFO:teuthology.orchestra.run.smithi063.stdout:1676 read 0xde6b5e0 thru 0xde6c4ce (0xeef bytes) 2024-09-13T09:26:25.100 INFO:teuthology.orchestra.run.smithi063.stdout:1677 read 0xa0f8a09 thru 0xa1060e8 (0xd6e0 bytes) 2024-09-13T09:26:25.101 INFO:teuthology.orchestra.run.smithi063.stdout:1678 writesame 0x8146227 thru 0x814dc28 data_size 0x17(0x7a02 bytes) 2024-09-13T09:26:25.116 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.116 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.117 INFO:teuthology.orchestra.run.smithi063.stdout:1679 writesame 0x98204c1 thru 0x982b3ef data_size 0x63(0xaf2f bytes) 2024-09-13T09:26:25.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:25.153 INFO:teuthology.orchestra.run.smithi063.stdout:1680 trunc from 0xe5c67df to 0x305ed73 2024-09-13T09:26:25.154 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:25.152+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:25.184 INFO:teuthology.orchestra.run.smithi063.stdout:1682 writesame 0x124a401 thru 0x125916b data_size 0x1(0xed6b bytes) 2024-09-13T09:26:25.211 INFO:teuthology.orchestra.run.smithi063.stdout:1685 write 0xc030fcd thru 0xc03ecb1 (0xdce5 bytes) 2024-09-13T09:26:25.228 INFO:teuthology.orchestra.run.smithi063.stdout:1686 read 0xc2a4 thru 0xe124 (0x1e81 bytes) 2024-09-13T09:26:25.229 INFO:teuthology.orchestra.run.smithi063.stdout:1687 read 0x92314f4 thru 0x9234dcd (0x38da bytes) 2024-09-13T09:26:25.229 INFO:teuthology.orchestra.run.smithi063.stdout:1688 read 0x910fd51 thru 0x9110447 (0x6f7 bytes) 2024-09-13T09:26:25.233 INFO:teuthology.orchestra.run.smithi063.stdout:1689 trunc from 0xc03ecb2 to 0xe1893a7 2024-09-13T09:26:25.245 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.245 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:25.256 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:25.251+0000 7f1615cdd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 22203) UID: 0 2024-09-13T09:26:25.264 INFO:teuthology.orchestra.run.smithi063.stdout:1691 trunc from 0xe1893a7 to 0x41b0f89 2024-09-13T09:26:25.312 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:26:25.312 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:25.312 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:26:25.312 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T09:26:25.312 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:26:25.313 INFO:teuthology.orchestra.run.smithi063.stdout:1693 write 0x9f0b7e1 thru 0x9f18a59 (0xd279 bytes) 2024-09-13T09:26:25.355 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.355 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.356 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:25.352+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:25.366 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:25.390 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T09:26:25.390 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-13T09:26:25.390 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:26:25.391 DEBUG:teuthology.orchestra.run.smithi063:> type ceph-objectstore-tool 2024-09-13T09:26:25.396 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:26:25.397 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-13T09:26:25.435 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:26:25.482 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.483 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.483 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:25.479+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:25.493 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:25.558 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:25.556+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:25.638 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.638 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:25.659 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:25.655+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:25.760 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:25.756+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:25.764 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.765 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.775 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:25.862 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:25.860+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:25.884 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:25.884 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:25.895 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:25.962 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:25.960+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:26.004 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.004 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:26.063 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:26.060+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:26.120 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.120 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.130 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:26.164 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:26.160+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:26.237 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.237 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:26.265 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:26.260+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:26.351 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.351 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:26.406 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:26.399+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:26.465 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.466 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.466 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:26.464+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:26.475 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:26.545 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:26.567 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:26.564+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.591 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.601 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:26.668 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:26.659+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:26.690 INFO:teuthology.orchestra.run.smithi063.stdout:3.b 2024-09-13T09:26:26.690 INFO:teuthology.orchestra.run.smithi063.stdout:3.7 2024-09-13T09:26:26.690 INFO:teuthology.orchestra.run.smithi063.stdout:3.5 2024-09-13T09:26:26.690 INFO:teuthology.orchestra.run.smithi063.stdout:3.8 2024-09-13T09:26:26.690 INFO:teuthology.orchestra.run.smithi063.stdout:3.1 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.1e 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.19 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.18 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.17 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.15 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.12 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.c 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:3.2 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:1.7 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:1.6 2024-09-13T09:26:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:1.4 2024-09-13T09:26:26.703 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:26.712 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:26.722 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:26.724 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:26.745 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:26.769 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:26.769 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:26.763+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:26.789 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:26.810 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:26.834 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:26.835 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:26.835 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3063 3.9 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:26.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.451317589999999, 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031119964999999999, 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3063 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382211+0000", 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382223+0000", 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2245e-05 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382253+0000", 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9947999999999999e-05 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382267+0000", 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4034e-05 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382338+0000", 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0931000000000002e-05 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382439+0000", 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001009 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.390414+0000", 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079749260000000002 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413261+0000", 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022847375 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413269+0000", 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:26:26.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413302+0000", 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3262000000000001e-05 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413333+0000", 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0077999999999999e-05 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22720 3.1b 3:d9bd9d03:::rbd_data.111811db2be7.00000000000000b7:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.6038539509999996, 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033396362999999998, 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22720 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229675+0000", 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:26.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229684+0000", 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229703+0000", 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9276999999999999e-05 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229713+0000", 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6979999999999993e-06 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229770+0000", 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7096999999999999e-05 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229856+0000", 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5376000000000004e-05 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.236948+0000", 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070919360000000001 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262783+0000", 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025835137000000001 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262895+0000", 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011265 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262993+0000", 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8052000000000002e-05 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.263073+0000", 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9244000000000002e-05 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25505 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5849088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954441+0000", 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.879088871, 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029315962000000001, 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25505 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954441+0000", 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954441+0000", 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954441+0000", 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954445+0000", 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5650000000000003e-06 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954454+0000", 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2039999999999997e-06 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954461+0000", 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:26:26.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983651+0000", 2024-09-13T09:26:26.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029189807000000002 2024-09-13T09:26:26.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983695+0000", 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4548000000000001e-05 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983757+0000", 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2225999999999997e-05 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25506 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5877760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.857 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954467+0000", 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.879063076, 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029497012, 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25506 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954467+0000", 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954467+0000", 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954467+0000", 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954470+0000", 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7179999999999998e-06 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954479+0000", 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.901e-06 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954487+0000", 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6170000000000001e-06 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983797+0000", 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029310411000000001 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983870+0000", 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3166999999999999e-05 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983964+0000", 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3733999999999994e-05 2024-09-13T09:26:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25507 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5906432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954507+0000", 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8790230719999998, 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029582375000000001, 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25507 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954507+0000", 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954507+0000", 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954506+0000", 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954511+0000", 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5569999999999997e-06 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954520+0000", 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3230000000000004e-06 2024-09-13T09:26:26.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954528+0000", 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0080000000000002e-06 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983985+0000", 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029456782000000001 2024-09-13T09:26:26.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984028+0000", 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3278999999999997e-05 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984089+0000", 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1422999999999998e-05 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25508 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5935104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8789956000000001, 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029767913999999999, 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25508 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954534+0000", 2024-09-13T09:26:26.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954538+0000", 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8010000000000001e-06 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954546+0000", 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5499999999999995e-06 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954553+0000", 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0049999999999998e-06 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984129+0000", 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029576203999999998 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984202+0000", 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000006e-05 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984303+0000", 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026600000000001 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25509 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5963776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8789711379999998, 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029860339, 2024-09-13T09:26:26.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25509 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954562+0000", 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3100000000000001e-06 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954571+0000", 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.816e-06 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954578+0000", 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-13T09:26:26.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984320+0000", 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029742527000000001 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984361+0000", 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0615999999999999e-05 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984419+0000", 2024-09-13T09:26:26.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8594000000000002e-05 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25510 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5992448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8789464169999999, 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030038194000000001, 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25510 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:26.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954583+0000", 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954587+0000", 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5640000000000001e-06 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954595+0000", 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.473e-06 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954602+0000", 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984458+0000", 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029856186 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984529+0000", 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0345000000000005e-05 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984622+0000", 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3197999999999998e-05 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25511 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6021120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8789217470000001, 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030144265, 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25511 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954632+0000", 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3645000000000001e-05 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954641+0000", 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4490000000000005e-06 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954649+0000", 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3750000000000003e-06 2024-09-13T09:26:26.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984638+0000", 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029988471999999999 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984678+0000", 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0170999999999999e-05 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984753+0000", 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4666999999999995e-05 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25512 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6033408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.878874105, 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030307508, 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25512 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:26.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954655+0000", 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954659+0000", 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954668+0000", 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954677+0000", 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1530000000000003e-06 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984792+0000", 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030114499999999999 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984866+0000", 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4437000000000006e-05 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984964+0000", 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7330000000000005e-05 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25513 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6062080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8788485660000003, 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030400765999999999, 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25513 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954681+0000", 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954687+0000", 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5039999999999999e-06 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954693+0000", 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0499999999999999e-06 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954700+0000", 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3089999999999998e-06 2024-09-13T09:26:26.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984980+0000", 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280509000000001 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985022+0000", 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1301000000000003e-05 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985082+0000", 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0726000000000001e-05 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25514 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6090752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.878824115, 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030608730000000001, 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25514 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:26.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954705+0000", 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954709+0000", 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7290000000000002e-06 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954718+0000", 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8049999999999996e-06 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954725+0000", 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985121+0000", 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030396636000000001 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985193+0000", 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1874999999999999e-05 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985315+0000", 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121452 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25515 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6119424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8787990489999999, 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030701790999999999, 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25515 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954730+0000", 2024-09-13T09:26:26.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954734+0000", 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8449999999999996e-06 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954744+0000", 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0222e-05 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954752+0000", 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2049999999999996e-06 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985334+0000", 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030582024999999999 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985375+0000", 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1253e-05 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985433+0000", 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7986999999999999e-05 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25516 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6148096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.878772632, 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030869497999999999, 2024-09-13T09:26:26.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25516 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954757+0000", 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954761+0000", 2024-09-13T09:26:26.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7479999999999999e-06 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954803+0000", 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2342000000000002e-05 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954811+0000", 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985473+0000", 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030661702999999998 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985541+0000", 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7800999999999997e-05 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985627+0000", 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6446000000000005e-05 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25517 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6176768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8787112480000001, 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030924971999999998, 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25517 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954818+0000", 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954823+0000", 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:26:26.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954834+0000", 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1174e-05 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954842+0000", 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0339999999999997e-06 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985645+0000", 2024-09-13T09:26:26.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030803298 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985687+0000", 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2089000000000003e-05 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985744+0000", 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6666999999999998e-05 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25518 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6205440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8786815809999999, 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031093369999999999, 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25518 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.913 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954848+0000", 2024-09-13T09:26:26.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954852+0000", 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8820000000000003e-06 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954860+0000", 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954868+0000", 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6680000000000001e-06 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985784+0000", 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03091648 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985852+0000", 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7407e-05 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985942+0000", 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25519 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6234112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.878655287, 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031185332, 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25519 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954874+0000", 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954878+0000", 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954886+0000", 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6209999999999993e-06 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954894+0000", 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3409999999999997e-06 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985960+0000", 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031066527 2024-09-13T09:26:26.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986002+0000", 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1925999999999998e-05 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986060+0000", 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7948999999999997e-05 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25520 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6262784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.878627351, 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031383412999999999, 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25520 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954902+0000", 2024-09-13T09:26:26.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954906+0000", 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8060000000000001e-06 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954914+0000", 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3059999999999998e-06 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954923+0000", 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7579999999999998e-06 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986106+0000", 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031182399 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986184+0000", 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8699e-05 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986286+0000", 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101977 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25563 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8785831320000002, 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034890003000000003, 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25563 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954946+0000", 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954953+0000", 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954963+0000", 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0468e-05 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954971+0000", 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2600000000000005e-06 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986090+0000", 2024-09-13T09:26:26.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031118930999999999 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986167+0000", 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6470999999999993e-05 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986448+0000", 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000281451 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989625+0000", 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003177352 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989740+0000", 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114632 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989798+0000", 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7555e-05 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989837+0000", 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9385999999999999e-05 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25577 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8736618419999997, 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032046094999999997, 2024-09-13T09:26:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25577 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959867+0000", 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959875+0000", 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9859999999999994e-06 2024-09-13T09:26:26.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959888+0000", 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2977e-05 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959898+0000", 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9280000000000005e-06 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986894+0000", 2024-09-13T09:26:26.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026995473999999998 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.987025+0000", 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013186200000000001 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.987422+0000", 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039672500000000001 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:26.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.990604+0000", 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00318207 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991786+0000", 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001181229 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991869+0000", 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3726000000000004e-05 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:26.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991914+0000", 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207000000000003e-05 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:26.938 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:26.939 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:26.940 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:26.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:26.941 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:26.936+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:27.040 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:27.035+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:27.055 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:27.055 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:27.055 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:27.055 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:27.065 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:27.141 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:27.135+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:27.176 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:27.176 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:27.177 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:27.177 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:27.177 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:27.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:27.242 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:27.239+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:27.267 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 3.8 --file /var/log/ceph/exp.3.8.3 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:27.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.82343564, 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:27.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:27.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:27.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:27.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:27.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.799093377999998, 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:27.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:27.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:27.305 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.794725015000001, 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:27.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:27.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:27.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.431296357000001, 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:27.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:27.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:27.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.594985875999999, 2024-09-13T09:26:27.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:27.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:27.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.341979847999999, 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:27.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:27.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29819 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:27.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8377427100000001, 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022753847000000001, 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29819 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450415+0000", 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450418+0000", 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450423+0000", 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450428+0000", 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463252+0000", 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012823467999999999 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463451+0000", 2024-09-13T09:26:27.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019896599999999999 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473170+0000", 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009718829 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29887 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:27.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.837495053, 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015606436, 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29887 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:27.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450665+0000", 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3640000000000002e-06 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450671+0000", 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450675+0000", 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:26:27.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466121+0000", 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015446051000000001 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466193+0000", 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1653e-05 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466270+0000", 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6756999999999997e-05 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29888 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.837467706, 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015745616, 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29888 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450690+0000", 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450693+0000", 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-06 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450699+0000", 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0009999999999998e-06 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450704+0000", 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466289+0000", 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015584242999999999 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466353+0000", 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4738000000000003e-05 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466436+0000", 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3151000000000004e-05 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29889 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.83712214, 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015595533999999999, 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29889 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:27.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451035+0000", 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451040+0000", 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451048+0000", 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7749999999999993e-06 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451054+0000", 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.314e-06 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466455+0000", 2024-09-13T09:26:27.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015401141 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466543+0000", 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7739999999999997e-05 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466632+0000", 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8672999999999996e-05 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29890 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8370985339999999, 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015753145, 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29890 2024-09-13T09:26:27.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451059+0000", 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451063+0000", 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451068+0000", 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6609999999999997e-06 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451073+0000", 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5120000000000002e-06 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466655+0000", 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015581858000000001 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466728+0000", 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2912999999999998e-05 2024-09-13T09:26:27.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466813+0000", 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5563999999999996e-05 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29891 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.354 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8370815469999999, 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015920059, 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29891 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451079+0000", 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451084+0000", 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3260000000000003e-06 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451089+0000", 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6380000000000003e-06 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466832+0000", 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015743549999999999 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466910+0000", 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7483000000000003e-05 2024-09-13T09:26:27.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466997+0000", 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7146999999999998e-05 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29921 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 90112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8362356580000001, 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015647616, 2024-09-13T09:26:27.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29921 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451922+0000", 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451925+0000", 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.334e-06 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451930+0000", 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9760000000000003e-06 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451935+0000", 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:26:27.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467403+0000", 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015468328 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467470+0000", 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6792999999999996e-05 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467570+0000", 2024-09-13T09:26:27.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100658 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29922 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8362177280000003, 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01583321, 2024-09-13T09:26:27.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29922 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451940+0000", 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451943+0000", 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7760000000000002e-06 2024-09-13T09:26:27.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451948+0000", 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451953+0000", 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467596+0000", 2024-09-13T09:26:27.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015643106 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467678+0000", 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2338999999999999e-05 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467774+0000", 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5891999999999996e-05 2024-09-13T09:26:27.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29923 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8362018469999999, 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016026886000000001, 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29923 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451956+0000", 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451969+0000", 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2823000000000001e-05 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451976+0000", 2024-09-13T09:26:27.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451981+0000", 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4650000000000002e-06 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467800+0000", 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015818412 2024-09-13T09:26:27.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467888+0000", 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8256000000000003e-05 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467984+0000", 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5680999999999994e-05 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29924 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 147456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8361730129999998, 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016203393999999999, 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29924 2024-09-13T09:26:27.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451988+0000", 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7920000000000001e-06 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451993+0000", 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2979999999999999e-06 2024-09-13T09:26:27.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451998+0000", 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.656e-06 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468003+0000", 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016005275999999999 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468093+0000", 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9530999999999994e-05 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468189+0000", 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6356999999999999e-05 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29925 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8361572690000001, 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016402086999999999, 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29925 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:27.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452003+0000", 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452008+0000", 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452012+0000", 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1749999999999998e-06 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468216+0000", 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016204309 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468307+0000", 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0917999999999999e-05 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468403+0000", 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6043000000000002e-05 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29926 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 204800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8361425229999999, 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016580355000000001, 2024-09-13T09:26:27.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29926 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452018+0000", 2024-09-13T09:26:27.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3099999999999999e-06 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452023+0000", 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4149999999999998e-06 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452028+0000", 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3730000000000003e-06 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468428+0000", 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016400237000000002 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468512+0000", 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4182000000000001e-05 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468596+0000", 2024-09-13T09:26:27.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169e-05 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29927 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.836126728, 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016758252000000001, 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29927 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:27.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452031+0000", 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452034+0000", 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452039+0000", 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.378e-06 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452043+0000", 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468625+0000", 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581722 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468701+0000", 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5909999999999997e-05 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468790+0000", 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8980999999999994e-05 2024-09-13T09:26:27.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:30010 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.833609467, 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015722454, 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 30010 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454551+0000", 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454556+0000", 2024-09-13T09:26:27.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454559+0000", 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3220000000000001e-06 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470137+0000", 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015577860000000001 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470210+0000", 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2986999999999998e-05 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470271+0000", 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1247999999999994e-05 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.397 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:27.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:27.400 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:27.395+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:27.451 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:26:27.464 INFO:teuthology.orchestra.run.smithi063.stdout:1694 write 0xde92a3e thru 0xde9c642 (0x9c05 bytes) 2024-09-13T09:26:27.478 INFO:teuthology.orchestra.run.smithi063.stdout:1695 write 0xb002ba0 thru 0xb00c453 (0x98b4 bytes) 2024-09-13T09:26:27.499 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:27.491+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:27.500 INFO:teuthology.orchestra.run.smithi063.stdout:1696 trunc from 0xde9c643 to 0x9525459 2024-09-13T09:26:27.518 INFO:teuthology.orchestra.run.smithi063.stdout:1697 write 0x9338b1b thru 0x9338e2e (0x314 bytes) 2024-09-13T09:26:27.521 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:27.524 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:27.524 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:27.525 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:27.533 INFO:teuthology.orchestra.run.smithi063.stdout:1700 writesame 0xa8d08c9 thru 0xa8d47da data_size 0x4e(0x3f12 bytes) 2024-09-13T09:26:27.533 INFO:teuthology.orchestra.run.smithi063.stdout:1702 write 0x80807e2 thru 0x808b578 (0xad97 bytes) 2024-09-13T09:26:27.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:27.545 INFO:teuthology.orchestra.run.smithi063.stdout:1703 punch from 0xa8c43d7 to 0xa8c83ff, (0x4028 bytes) 2024-09-13T09:26:27.546 INFO:teuthology.orchestra.run.smithi063.stdout:1704 write 0xa1fab9d thru 0xa1fca0b (0x1e6f bytes) 2024-09-13T09:26:27.554 INFO:teuthology.orchestra.run.smithi063.stdout:1705 write 0x6b9dca2 thru 0x6ba05d5 (0x2934 bytes) 2024-09-13T09:26:27.565 INFO:teuthology.orchestra.run.smithi063.stdout:1706 writesame 0x54da9fc thru 0x54e4dc6 data_size 0x9(0xa3cb bytes) 2024-09-13T09:26:27.581 INFO:teuthology.orchestra.run.smithi063.stdout:1707 punch from 0x2c7ced2 to 0x2c88dc8, (0xbef6 bytes) 2024-09-13T09:26:27.592 INFO:teuthology.orchestra.run.smithi063.stdout:1709 write 0xd3d821c thru 0xd3ddad3 (0x58b8 bytes) 2024-09-13T09:26:27.600 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:27.596+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:27.605 INFO:teuthology.orchestra.run.smithi063.stdout:1710 punch from 0x18ab99a to 0x18b9161, (0xd7c7 bytes) 2024-09-13T09:26:27.605 INFO:teuthology.orchestra.run.smithi063.stdout:1711 write 0x7502438 thru 0x750517d (0x2d46 bytes) 2024-09-13T09:26:27.613 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:27.613 INFO:teuthology.orchestra.run.smithi063.stdout:1713 read 0x26d27e3 thru 0x26d9214 (0x6a32 bytes) 2024-09-13T09:26:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:1716 punch from 0x78a4b31 to 0x78b178d, (0xcc5c bytes) 2024-09-13T09:26:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:1717 punch from 0x808d16a to 0x8099aa0, (0xc936 bytes) 2024-09-13T09:26:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:1718 punch from 0x7ec17c8 to 0x7ed0915, (0xf14d bytes) 2024-09-13T09:26:27.615 INFO:teuthology.orchestra.run.smithi063.stdout:1719 punch from 0x9f3884d to 0x9f43160, (0xa913 bytes) 2024-09-13T09:26:27.625 INFO:teuthology.orchestra.run.smithi063.stdout:1720 trunc from 0xd3ddad4 to 0x7cc1153 2024-09-13T09:26:27.639 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:27.640 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:27.640 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:27.640 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:27.640 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:27.645 INFO:teuthology.orchestra.run.smithi063.stdout:1721 write 0x77e421 thru 0x780af3 (0x26d3 bytes) 2024-09-13T09:26:27.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:27.654 INFO:teuthology.orchestra.run.smithi063.stdout:1723 trunc from 0x7cc1153 to 0x98cad6e 2024-09-13T09:26:27.667 INFO:teuthology.orchestra.run.smithi063.stdout:1724 trunc from 0x98cad6e to 0x826fc07 2024-09-13T09:26:27.678 INFO:teuthology.orchestra.run.smithi063.stdout:1726 writesame 0x5384f66 thru 0x5393e13 data_size 0x2(0xeeae bytes) 2024-09-13T09:26:27.698 INFO:teuthology.orchestra.run.smithi063.stdout:1727 read 0x4a23567 thru 0x4a29ca9 (0x6743 bytes) 2024-09-13T09:26:27.698 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:27.698 INFO:teuthology.orchestra.run.smithi063.stdout:1731 read 0x40c5ba0 thru 0x40cc992 (0x6df3 bytes) 2024-09-13T09:26:27.699 INFO:teuthology.orchestra.run.smithi063.stdout:1732 punch from 0x38b7033 to 0x38c2440, (0xb40d bytes) 2024-09-13T09:26:27.699 INFO:teuthology.orchestra.run.smithi063.stdout:1733 read 0x295473a thru 0x295b75c (0x7023 bytes) 2024-09-13T09:26:27.701 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:27.696+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:27.702 INFO:teuthology.orchestra.run.smithi063.stdout:1735 trunc from 0x826fc07 to 0x683dd4d 2024-09-13T09:26:27.716 INFO:teuthology.orchestra.run.smithi063.stdout:1738 write 0x5c90391 thru 0x5c92fe1 (0x2c51 bytes) 2024-09-13T09:26:27.729 INFO:teuthology.orchestra.run.smithi063.stdout:1739 read 0xd9b2bf thru 0xda3fa6 (0x8ce8 bytes) 2024-09-13T09:26:27.730 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:27.730 INFO:teuthology.orchestra.run.smithi063.stdout:1742 read 0x286f049 thru 0x287dd80 (0xed38 bytes) 2024-09-13T09:26:27.731 INFO:teuthology.orchestra.run.smithi063.stdout:1743 clone 14 order 23 su 2097152 sc 5 2024-09-13T09:26:27.798 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.236306124999999, 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29295 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.35402215, 2024-09-13T09:26:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031712789999999998, 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29295 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441416+0000", 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441422+0000", 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.592e-06 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441427+0000", 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.453205+0000", 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011778116999999999 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454881+0000", 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016761879999999999 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473127+0000", 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018246294 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29497 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3509762009999999, 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024159292999999998, 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29497 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444462+0000", 2024-09-13T09:26:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5129999999999999e-06 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444468+0000", 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4009999999999996e-06 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444475+0000", 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8990000000000001e-06 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.465234+0000", 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020758316999999998 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.465299+0000", 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4873e-05 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468620+0000", 2024-09-13T09:26:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033208769999999999 2024-09-13T09:26:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29498 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3509569849999998, 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024172662000000001, 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29498 2024-09-13T09:26:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444481+0000", 2024-09-13T09:26:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444486+0000", 2024-09-13T09:26:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1070000000000004e-06 2024-09-13T09:26:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444491+0000", 2024-09-13T09:26:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2379999999999999e-06 2024-09-13T09:26:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468645+0000", 2024-09-13T09:26:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024153813999999999 2024-09-13T09:26:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468652+0000", 2024-09-13T09:26:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6290000000000001e-06 2024-09-13T09:26:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29499 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3509419789999999, 2024-09-13T09:26:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024184213, 2024-09-13T09:26:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29499 2024-09-13T09:26:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444496+0000", 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444502+0000", 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444506+0000", 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.279e-06 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468672+0000", 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02416565 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468679+0000", 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9959999999999996e-06 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29500 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3509236480000002, 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024191877000000001, 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29500 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444515+0000", 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1909999999999999e-06 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444520+0000", 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6230000000000002e-06 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444525+0000", 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5040000000000004e-06 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468698+0000", 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024173310999999999 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468705+0000", 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.545e-06 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29501 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.350907871, 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024199919, 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29501 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444528+0000", 2024-09-13T09:26:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444531+0000", 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.311e-06 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444536+0000", 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1880000000000001e-06 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444540+0000", 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4009999999999999e-06 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468722+0000", 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024182209 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468729+0000", 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29502 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3508924520000001, 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024207818999999998, 2024-09-13T09:26:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29502 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444549+0000", 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4929999999999998e-06 2024-09-13T09:26:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444554+0000", 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444559+0000", 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0980000000000004e-06 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468745+0000", 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024186736 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468752+0000", 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5220000000000004e-06 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29503 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.350873977, 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024211703000000001, 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29503 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444564+0000", 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000002e-06 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444570+0000", 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4550000000000003e-06 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444574+0000", 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4129999999999999e-06 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468768+0000", 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024193840000000001 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468774+0000", 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29504 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.350858465, 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024219068999999999, 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29504 2024-09-13T09:26:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444580+0000", 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444585+0000", 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4260000000000002e-06 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444590+0000", 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2289999999999997e-06 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468791+0000", 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024201139999999999 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468797+0000", 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29542 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3508430589999998, 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024226968000000002, 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29542 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444601+0000", 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444607+0000", 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9660000000000001e-06 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444611+0000", 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6920000000000001e-06 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468814+0000", 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024202893 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468820+0000", 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29544 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3505105959999999, 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023918539999999999, 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29544 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444925+0000", 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444933+0000", 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3640000000000001e-06 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444941+0000", 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9449999999999993e-06 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444947+0000", 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468838+0000", 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023891194000000001 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468844+0000", 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3380000000000001e-06 2024-09-13T09:26:27.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29545 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3504849760000002, 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023916804, 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29545 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:27.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444954+0000", 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.615e-06 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444959+0000", 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444964+0000", 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468862+0000", 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023898189 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468868+0000", 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4330000000000001e-06 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29546 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3504690510000001, 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023923954000000001, 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29546 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:27.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444969+0000", 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444975+0000", 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4940000000000003e-06 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444979+0000", 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5379999999999996e-06 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468885+0000", 2024-09-13T09:26:27.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023905531000000001 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468891+0000", 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.455e-06 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29547 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3504531690000001, 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023930462, 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29547 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444985+0000", 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3290000000000001e-06 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444992+0000", 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709999999999996e-06 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444997+0000", 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3409999999999996e-06 2024-09-13T09:26:27.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468907+0000", 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023910579000000001 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468914+0000", 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4019999999999999e-06 2024-09-13T09:26:27.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29548 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3504357759999999, 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023936915, 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29548 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445000+0000", 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445003+0000", 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:26:27.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445008+0000", 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445017+0000", 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.533e-06 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468931+0000", 2024-09-13T09:26:27.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023913750000000001 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468938+0000", 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29549 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:27.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3504145100000002, 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023939946, 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29549 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445024+0000", 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4930000000000001e-06 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445030+0000", 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.088e-06 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445035+0000", 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468955+0000", 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023920675999999998 2024-09-13T09:26:27.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468962+0000", 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29550 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3503978000000001, 2024-09-13T09:26:27.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023946261999999999, 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29550 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445038+0000", 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445041+0000", 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3329999999999999e-06 2024-09-13T09:26:27.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445046+0000", 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445051+0000", 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5149999999999997e-06 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468979+0000", 2024-09-13T09:26:27.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023927166 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468985+0000", 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29551 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.891 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3503663420000001, 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023937653, 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29551 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:27.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445073+0000", 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0070000000000002e-06 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445079+0000", 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445084+0000", 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8169999999999998e-06 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.469002+0000", 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023917559000000001 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.469008+0000", 2024-09-13T09:26:27.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1319999999999999e-06 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29666 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.448198+0000", 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.34723828, 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023862116999999999, 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:27.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29666 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.448198+0000", 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.448198+0000", 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:27.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.448198+0000", 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.448200+0000", 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:27.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.448229+0000", 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8546000000000001e-05 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.448246+0000", 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6908000000000001e-05 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.472054+0000", 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023808132999999999 2024-09-13T09:26:27.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.472060+0000", 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.472e-06 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:27.900 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:27.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:27.902 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:27.895+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:28.003 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:27.999+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:28.015 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:28.016 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:28.016 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:28.025 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:28.104 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:28.099+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:28.131 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:28.131 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:28.131 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:28.131 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:28.131 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:28.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:28.205 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:28.195+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:28.244 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.280209663000001, 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 44.283820208000002, 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:4479 3.d 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.405464243000001, 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051456631000000003, 2024-09-13T09:26:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4479 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835751+0000", 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999909 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854271+0000", 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01851972 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854305+0000", 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4258000000000003e-05 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854317+0000", 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2013e-05 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854487+0000", 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017018 2024-09-13T09:26:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854768+0000", 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028067700000000001 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.855666+0000", 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000898404 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.881583+0000", 2024-09-13T09:26:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025916947999999999 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887097+0000", 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055134479999999998 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887161+0000", 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4077000000000003e-05 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887217+0000", 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6128000000000002e-05 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29813 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 417792~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475915+0000", 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7653097300000002, 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041775513, 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29813 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475915+0000", 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475915+0000", 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475914+0000", 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475917+0000", 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.79e-06 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475923+0000", 2024-09-13T09:26:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9020000000000004e-06 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475928+0000", 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517554+0000", 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041626412000000002 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517612+0000", 2024-09-13T09:26:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7411999999999999e-05 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517690+0000", 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8405999999999994e-05 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29814 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 446464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7652922699999998, 2024-09-13T09:26:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041923992, 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29814 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475934+0000", 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4159999999999998e-06 2024-09-13T09:26:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475939+0000", 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475943+0000", 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517722+0000", 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041778932999999997 2024-09-13T09:26:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517777+0000", 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4843999999999999e-05 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517856+0000", 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8813999999999995e-05 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29815 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 475136~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7652769369999999, 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042062642999999997, 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29815 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475949+0000", 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1830000000000001e-06 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475954+0000", 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1220000000000003e-06 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475960+0000", 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9660000000000001e-06 2024-09-13T09:26:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517875+0000", 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041914712999999999 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517932+0000", 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7411000000000003e-05 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518010+0000", 2024-09-13T09:26:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7585999999999999e-05 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29816 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 503808~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7652600659999997, 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042212265999999998, 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29816 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475966+0000", 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.165e-06 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475971+0000", 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0409999999999997e-06 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475976+0000", 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518042+0000", 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042066192000000002 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518098+0000", 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6347000000000001e-05 2024-09-13T09:26:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518177+0000", 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8227999999999997e-05 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29871 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 20480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475980+0000", 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7652447929999999, 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04234835, 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29871 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475980+0000", 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475980+0000", 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475979+0000", 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475999+0000", 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9633e-05 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476005+0000", 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2740000000000004e-06 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476010+0000", 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518195+0000", 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042184455000000003 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518252+0000", 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7627000000000002e-05 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518328+0000", 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5717999999999996e-05 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29873 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.765210191, 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042371669000000001, 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29873 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476016+0000", 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3769999999999999e-06 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476021+0000", 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476027+0000", 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2049999999999999e-06 2024-09-13T09:26:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518371+0000", 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042343486 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518386+0000", 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5121e-05 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29874 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7651929580000001, 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042386539000000001, 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29874 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476033+0000", 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.289e-06 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476039+0000", 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476043+0000", 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:26:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518408+0000", 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042364786000000001 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518418+0000", 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7010000000000008e-06 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29875 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7651722759999999, 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042388589999999997, 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29875 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476054+0000", 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476060+0000", 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.356e-06 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476066+0000", 2024-09-13T09:26:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4489999999999996e-06 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518433+0000", 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042366899999999999 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518441+0000", 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7980000000000006e-06 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29880 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7643440369999999, 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041772767000000002, 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29880 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476883+0000", 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476889+0000", 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1569999999999997e-06 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476908+0000", 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9015999999999999e-05 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518646+0000", 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041737784 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518653+0000", 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5569999999999998e-06 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29892 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476926+0000", 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7642984560000001, 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041808019000000002, 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29892 2024-09-13T09:26:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476926+0000", 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476926+0000", 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476925+0000", 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476928+0000", 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1039999999999998e-06 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476936+0000", 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1500000000000002e-06 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476957+0000", 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1090999999999998e-05 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518701+0000", 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041744548999999999 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518734+0000", 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2641999999999997e-05 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29893 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7642620139999998, 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041786809000000001, 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29893 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476971+0000", 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1749999999999994e-06 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476979+0000", 2024-09-13T09:26:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1499999999999999e-06 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476984+0000", 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2830000000000001e-06 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518745+0000", 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041760799000000001 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518749+0000", 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9369999999999997e-06 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29894 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.7642351830000003, 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041820164999999999, 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29894 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476991+0000", 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6029999999999999e-06 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477000+0000", 2024-09-13T09:26:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1219999999999995e-06 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477006+0000", 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518795+0000", 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041789735000000001 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518809+0000", 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4038e-05 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29895 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477010+0000", 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.764214752, 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041816114000000001, 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29895 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477010+0000", 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477010+0000", 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477009+0000", 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477012+0000", 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.26e-06 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477018+0000", 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477023+0000", 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518822+0000", 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041798163999999999 2024-09-13T09:26:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518826+0000", 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29896 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.764197174, 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041844904000000002, 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29896 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477029+0000", 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:26:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477034+0000", 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9949999999999996e-06 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477038+0000", 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2969999999999997e-06 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518849+0000", 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041810711 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518872+0000", 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2909e-05 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29897 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.764181904, 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856206, 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29897 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477050+0000", 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0679999999999997e-06 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477056+0000", 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.57e-06 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477060+0000", 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1810000000000003e-06 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518891+0000", 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041830621999999998 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518899+0000", 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0919999999999998e-06 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29898 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477065+0000", 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.764159663, 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041867695000000003, 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29898 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477065+0000", 2024-09-13T09:26:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477065+0000", 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477064+0000", 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477072+0000", 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2579999999999998e-06 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477077+0000", 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.536e-06 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477083+0000", 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7710000000000003e-06 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518924+0000", 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041840914 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518932+0000", 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5429999999999994e-06 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29899 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477103+0000", 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.764121351, 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041865687999999998, 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29899 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477103+0000", 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477103+0000", 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477102+0000", 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477105+0000", 2024-09-13T09:26:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.05e-06 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477111+0000", 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8200000000000002e-06 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477116+0000", 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5029999999999999e-06 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518955+0000", 2024-09-13T09:26:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041839297999999997 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518969+0000", 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3682999999999999e-05 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:28.332 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:28.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:28.334 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:28.327+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:28.434 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:28.432+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:28.453 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:28.453 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:28.453 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:28.465 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:28.467 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:26:28.536 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:28.532+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:28.572 INFO:teuthology.orchestra.run.smithi063.stderr:Exporting 3.8 info 3.8( v 46'129 (0'0,46'129] local-lis/les=21/22 n=16 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-13T09:26:28.582 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:28.582 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:28.582 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:28.583 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:28.583 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:28.583 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:111884c9:::rbd_data.1118e3e5e25d.00000000000000bf:head# 2024-09-13T09:26:28.584 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:121fe3f4:::rbd_data.1118a28cd5c4.0000000000000007:head# 2024-09-13T09:26:28.587 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:1269e70a:::rbd_data.1118a28cd5c4.0000000000000009:head# 2024-09-13T09:26:28.590 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:127fce81:::rbd_data.111851704b2b.0000000000000001:9# 2024-09-13T09:26:28.591 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:127fce81:::rbd_data.111851704b2b.0000000000000001:head# 2024-09-13T09:26:28.592 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:13ad7219:::rbd_data.1118a28cd5c4.0000000000000032:head# 2024-09-13T09:26:28.592 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:1480b51f:::rbd_data.1118a28cd5c4.0000000000000054:head# 2024-09-13T09:26:28.594 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:28.594 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:15cb164d:::rbd_data.1118a28cd5c4.00000000000000d3:head# 2024-09-13T09:26:28.595 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:16185ba7:::rbd_data.111867511a81.0000000000000009:5# 2024-09-13T09:26:28.596 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:16185ba7:::rbd_data.111867511a81.0000000000000009:head# 2024-09-13T09:26:28.596 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:163352f1:::rbd_object_map.11189888a71f:head# 2024-09-13T09:26:28.597 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:1718b63c:::rbd_data.1118e3e5e25d.00000000000000bc:head# 2024-09-13T09:26:28.597 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:173d6c83:::rbd_data.1118e98b86fc.0000000000000013:d# 2024-09-13T09:26:28.598 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:173d6c83:::rbd_data.1118e98b86fc.0000000000000013:head# 2024-09-13T09:26:28.598 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:1752c46c:::rbd_data.1118a28cd5c4.0000000000000000:head# 2024-09-13T09:26:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:Read #3:17e81ee6:::rbd_id.image_client.0-clone4:head# 2024-09-13T09:26:28.600 INFO:teuthology.orchestra.run.smithi063.stderr:Export successful 2024-09-13T09:26:28.636 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:28.627+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:28.704 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.750981996, 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:28.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:28.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.734423782999997, 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:28.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.499676524999998, 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:28.713 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:28.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:28.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.497467387999997, 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:28.719 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:28.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.722 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.493122863000004, 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.723 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:28.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.476571884000002, 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:28.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.475983876000001, 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:28.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:28.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.468277720000003, 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:28.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:28.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:28.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:28.741 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:28.741 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:28.742 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T09:26:28.742 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:28.743 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:28.740+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:28.843 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:28.840+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:28.865 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:28.865 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:28.866 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:28.866 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:28.876 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:28.941 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:28.944 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:28.935+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:28.982 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:28.983 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:28.993 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:29.011 INFO:teuthology.orchestra.run.smithi063.stdout:1744 punch from 0x394f659 to 0x3954931, (0x52d8 bytes) 2024-09-13T09:26:29.019 INFO:teuthology.orchestra.run.smithi063.stdout:1746 trunc from 0x683dd4d to 0x54e2408 2024-09-13T09:26:29.039 INFO:teuthology.orchestra.run.smithi063.stdout:1747 read 0x178bd06 thru 0x17935e6 (0x78e1 bytes) 2024-09-13T09:26:29.044 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:29.035+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:29.051 INFO:teuthology.orchestra.run.smithi063.stdout:1748 trunc from 0x54e2408 to 0xc29a27a 2024-09-13T09:26:29.059 INFO:teuthology.orchestra.run.smithi063.stdout:1750 write 0x66fc59b thru 0x6701b88 (0x55ee bytes) 2024-09-13T09:26:29.062 INFO:teuthology.orchestra.run.smithi063.stdout:1753 read 0xa85d7c1 thru 0xa8683f0 (0xac30 bytes) 2024-09-13T09:26:29.062 INFO:teuthology.orchestra.run.smithi063.stdout:1756 writesame 0x26b883c thru 0x26bb7f7 data_size 0x41(0x2fbc bytes) 2024-09-13T09:26:29.086 INFO:teuthology.orchestra.run.smithi063.stdout:1757 write 0xe6c458a thru 0xe6d3363 (0xedda bytes) 2024-09-13T09:26:29.086 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:29.092 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:29.092 INFO:teuthology.orchestra.run.smithi063.stdout:1760 read 0x95b12d9 thru 0x95bf201 (0xdf29 bytes) 2024-09-13T09:26:29.093 INFO:teuthology.orchestra.run.smithi063.stdout:1761 write 0xc02197d thru 0xc030130 (0xe7b4 bytes) 2024-09-13T09:26:29.097 INFO:teuthology.orchestra.run.smithi063.stdout:1764 write 0xbf2ef77 thru 0xbf3627d (0x7307 bytes) 2024-09-13T09:26:29.102 INFO:teuthology.orchestra.run.smithi063.stdout:1765 write 0x962f0b9 thru 0x9631e2a (0x2d72 bytes) 2024-09-13T09:26:29.106 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:29.107 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.107 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.107 INFO:teuthology.orchestra.run.smithi063.stdout:1766 read 0x92e19bf thru 0x92ef1e4 (0xd826 bytes) 2024-09-13T09:26:29.107 INFO:teuthology.orchestra.run.smithi063.stdout:1767 writesame 0x9d634a thru 0x9d9d1c data_size 0x1(0x39d3 bytes) 2024-09-13T09:26:29.117 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:29.118 INFO:teuthology.orchestra.run.smithi063.stdout:1769 read 0xb7af685 thru 0xb7ba01e (0xa99a bytes) 2024-09-13T09:26:29.118 INFO:teuthology.orchestra.run.smithi063.stdout:1770 read 0xbde929e thru 0xbdf10e4 (0x7e47 bytes) 2024-09-13T09:26:29.118 INFO:teuthology.orchestra.run.smithi063.stdout:1771 write 0x89f17b thru 0x8abb85 (0xca0b bytes) 2024-09-13T09:26:29.120 INFO:teuthology.orchestra.run.smithi063.stdout:1772 read 0xdeae1 thru 0xe1cda (0x31fa bytes) 2024-09-13T09:26:29.122 INFO:teuthology.orchestra.run.smithi063.stdout:1773 write 0x99ae2cd thru 0x99b1a57 (0x378b bytes) 2024-09-13T09:26:29.126 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:29.129 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:29.129 INFO:teuthology.orchestra.run.smithi063.stdout:1775 writesame 0x5964092 thru 0x596fb2c data_size 0x1f(0xba9b bytes) 2024-09-13T09:26:29.145 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:29.135+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:29.158 INFO:teuthology.orchestra.run.smithi063.stdout:1776 trunc from 0xe6d3364 to 0x13b894b 2024-09-13T09:26:29.167 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:29.200 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:29.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 3.8 2024-09-13T09:26:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:1777 writesame 0xd3222c6 thru 0xd332292 data_size 0x7(0xffcd bytes) 2024-09-13T09:26:29.212 INFO:teuthology.orchestra.run.smithi063.stdout:1778 writesame 0x9affab thru 0x9b3e97 data_size 0x1(0x3eed bytes) 2024-09-13T09:26:29.222 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:29.223 INFO:teuthology.orchestra.run.smithi063.stdout:1779 read 0x9ba7641 thru 0x9badf25 (0x68e5 bytes) 2024-09-13T09:26:29.223 INFO:teuthology.orchestra.run.smithi063.stdout:1782 writesame 0x19eaaa1 thru 0x19f2b93 data_size 0x1(0x80f3 bytes) 2024-09-13T09:26:29.236 INFO:teuthology.orchestra.run.smithi063.stdout:1785 write 0x68c7668 thru 0x68c9b79 (0x2512 bytes) 2024-09-13T09:26:29.241 INFO:teuthology.orchestra.run.smithi063.stdout:1786 write 0xb4a6903 thru 0xb4aff77 (0x9675 bytes) 2024-09-13T09:26:29.242 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.242 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.244 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:29.244 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:29.245 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:29.246 INFO:teuthology.orchestra.run.smithi063.stdout:1787 read 0x1eea73c thru 0x1eeb1ed (0xab2 bytes) 2024-09-13T09:26:29.246 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:29.239+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:29.247 INFO:teuthology.orchestra.run.smithi063.stdout:1789 punch from 0x642b78e to 0x642c4b7, (0xd29 bytes) 2024-09-13T09:26:29.247 INFO:teuthology.orchestra.run.smithi063.stdout:1790 write 0x85096b3 thru 0x851688f (0xd1dd bytes) 2024-09-13T09:26:29.248 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:29.249 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:29.252 INFO:teuthology.orchestra.run.smithi063.stdout:1791 write 0x4cf5386 thru 0x4cfe5ee (0x9269 bytes) 2024-09-13T09:26:29.257 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:29.260 INFO:teuthology.orchestra.run.smithi063.stdout:1795 read 0x5c5d9d2 thru 0x5c69371 (0xb9a0 bytes) 2024-09-13T09:26:29.260 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:29.260 INFO:teuthology.orchestra.run.smithi063.stdout:1797 write 0x567c5b0 thru 0x568a744 (0xe195 bytes) 2024-09-13T09:26:29.265 INFO:teuthology.orchestra.run.smithi063.stdout:1798 read 0x7fb4b8b thru 0x7fb60e7 (0x155d bytes) 2024-09-13T09:26:29.265 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:29.265 INFO:teuthology.orchestra.run.smithi063.stdout:1801 write 0x4e37792 thru 0x4e46010 (0xe87f bytes) 2024-09-13T09:26:29.270 INFO:teuthology.orchestra.run.smithi063.stdout:1802 clone 15 order 22 su 16384 sc 12 2024-09-13T09:26:29.347 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:29.344+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:29.357 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.357 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.367 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:29.449 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:29.439+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:29.527 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.527 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.537 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:29.549 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:29.539+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:29.595 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone14 from 0xd332293 (overlap 0x13b894b) to 0x0 2024-09-13T09:26:29.649 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:29.643+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:29.657 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.657 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:29.701 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T09:26:29.750 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:29.743+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:29.771 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.772 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:29.851 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:29.843+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:29.884 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:29.885 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:29.894 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:29.912 INFO:teuthology.orchestra.run.smithi063.stdout:1803 read 0x22c9ee1 thru 0x22d2aba (0x8bda bytes) 2024-09-13T09:26:29.926 INFO:teuthology.orchestra.run.smithi063.stdout:1804 read 0xa76a329 thru 0xa76c008 (0x1ce0 bytes) 2024-09-13T09:26:29.930 INFO:teuthology.orchestra.run.smithi063.stdout:1805 write 0x1464bf7 thru 0x1469e2c (0x5236 bytes) 2024-09-13T09:26:29.952 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:29.948+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:30.035 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.035 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:30.053 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:30.043+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:30.069 INFO:teuthology.orchestra.run.smithi063.stdout:1806 read 0x3d91bab thru 0x3d9d734 (0xbb8a bytes) 2024-09-13T09:26:30.075 INFO:teuthology.orchestra.run.smithi063.stdout:1807 writesame 0x9b3efbc thru 0x9b40517 data_size 0x2(0x155c bytes) 2024-09-13T09:26:30.096 INFO:teuthology.orchestra.run.smithi063.stdout:1808 punch from 0x281b959 to 0x2823cef, (0x8396 bytes) 2024-09-13T09:26:30.096 INFO:teuthology.orchestra.run.smithi063.stdout:1811 write 0xaaf71dc thru 0xaafc31c (0x5141 bytes) 2024-09-13T09:26:30.112 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.137 INFO:teuthology.orchestra.run.smithi063.stdout:1813 trunc from 0xd332293 to 0x1e73206 2024-09-13T09:26:30.153 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:30.147+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:30.176 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.176 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:30.254 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:30.252+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:30.304 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.304 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:30.361 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:30.360+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:30.389 INFO:teuthology.orchestra.run.smithi063.stdout: marking collection for removal 2024-09-13T09:26:30.389 INFO:teuthology.orchestra.run.smithi063.stdout:setting '_remove' omap key 2024-09-13T09:26:30.389 INFO:teuthology.orchestra.run.smithi063.stdout:finish_remove_pgs 3.8_head removing 3.8 2024-09-13T09:26:30.397 INFO:teuthology.orchestra.run.smithi063.stdout:Remove successful 2024-09-13T09:26:30.423 INFO:teuthology.orchestra.run.smithi063.stdout:1814 writesame 0x136f0c2 thru 0x137a635 data_size 0x31(0xb574 bytes) 2024-09-13T09:26:30.426 INFO:teuthology.orchestra.run.smithi063.stdout:1817 read 0x16cd012 thru 0x16d585c (0x884b bytes) 2024-09-13T09:26:30.428 INFO:teuthology.orchestra.run.smithi063.stdout:1818 trunc from 0x1e73206 to 0x1cdf0ea 2024-09-13T09:26:30.442 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.456 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:30.447+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:30.469 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.469 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.473 INFO:teuthology.orchestra.run.smithi063.stdout:1820 writesame 0xa6684a9 thru 0xa66b7e5 data_size 0x1(0x333d bytes) 2024-09-13T09:26:30.473 INFO:teuthology.orchestra.run.smithi063.stdout:1821 read 0x23d6a87 thru 0x23dc2ba (0x5834 bytes) 2024-09-13T09:26:30.474 INFO:teuthology.orchestra.run.smithi063.stdout:1822 read 0x3ebedd3 thru 0x3eca491 (0xb6bf bytes) 2024-09-13T09:26:30.474 INFO:teuthology.orchestra.run.smithi063.stdout:1823 writesame 0x9d8e238 thru 0x9d977b4 data_size 0xb(0x957d bytes) 2024-09-13T09:26:30.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:30.483 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:26:30.483 INFO:teuthology.orchestra.run.smithi063.stdout:1824 punch from 0xa5d8cca to 0xa5e4298, (0xb5ce bytes) 2024-09-13T09:26:30.483 INFO:teuthology.orchestra.run.smithi063.stdout:1825 read 0x6b03f09 thru 0x6b040b7 (0x1af bytes) 2024-09-13T09:26:30.484 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.499 INFO:teuthology.orchestra.run.smithi063.stdout:1827 write 0xc920cd3 thru 0xc92c322 (0xb650 bytes) 2024-09-13T09:26:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:1828 write 0xe692f1e thru 0xe69c711 (0x97f4 bytes) 2024-09-13T09:26:30.529 INFO:teuthology.orchestra.run.smithi063.stdout:1829 read 0xa6b1c43 thru 0xa6bd6a4 (0xba62 bytes) 2024-09-13T09:26:30.529 INFO:teuthology.orchestra.run.smithi063.stdout:1830 write 0x5926096 thru 0x593186c (0xb7d7 bytes) 2024-09-13T09:26:30.538 INFO:teuthology.orchestra.run.smithi063.stdout:1831 read 0x4acbb21 thru 0x4ad3664 (0x7b44 bytes) 2024-09-13T09:26:30.538 INFO:teuthology.orchestra.run.smithi063.stdout:1833 write 0x79afba7 thru 0x79bb98d (0xbde7 bytes) 2024-09-13T09:26:30.550 INFO:teuthology.orchestra.run.smithi063.stdout:1836 write 0x3a7f34 thru 0x3ab33d (0x340a bytes) 2024-09-13T09:26:30.556 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:30.552+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:30.570 INFO:teuthology.orchestra.run.smithi063.stdout:1837 trunc from 0xe69c712 to 0x5cbd5ff 2024-09-13T09:26:30.591 INFO:teuthology.orchestra.run.smithi063.stdout:1838 write 0x3120cec thru 0x3121d55 (0x106a bytes) 2024-09-13T09:26:30.596 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.596 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.606 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:30.609 INFO:teuthology.orchestra.run.smithi063.stdout:1840 write 0x8b3c94d thru 0x8b4877e (0xbe32 bytes) 2024-09-13T09:26:30.617 INFO:teuthology.orchestra.run.smithi063.stdout:1843 writesame 0xc9dff2 thru 0xcacf33 data_size 0x7d(0xef42 bytes) 2024-09-13T09:26:30.630 INFO:teuthology.orchestra.run.smithi063.stdout:1844 trunc from 0x8b4877f to 0x2816ede 2024-09-13T09:26:30.650 INFO:teuthology.orchestra.run.smithi063.stdout:1845 read 0x134a19d thru 0x13503ee (0x6252 bytes) 2024-09-13T09:26:30.657 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:30.656+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:30.675 INFO:teuthology.orchestra.run.smithi063.stdout:1846 write 0x97f8800 thru 0x97fdb6d (0x536e bytes) 2024-09-13T09:26:30.681 INFO:teuthology.orchestra.run.smithi063.stdout:1849 read 0x5306c1c thru 0x5310ae2 (0x9ec7 bytes) 2024-09-13T09:26:30.686 INFO:teuthology.orchestra.run.smithi063.stdout:1852 trunc from 0x97fdb6e to 0x6ced795 2024-09-13T09:26:30.701 INFO:teuthology.orchestra.run.smithi063.stdout:1854 write 0x4552d7e thru 0x455f2ec (0xc56f bytes) 2024-09-13T09:26:30.717 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.717 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.727 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:30.734 INFO:teuthology.orchestra.run.smithi063.stdout:1857 write 0xb2d483b thru 0xb2dc0fe (0x78c4 bytes) 2024-09-13T09:26:30.741 INFO:teuthology.orchestra.run.smithi063.stdout:1858 writesame 0x199495d thru 0x19a03d8 data_size 0x8c(0xba7c bytes) 2024-09-13T09:26:30.744 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.745 INFO:teuthology.orchestra.run.smithi063.stdout:1860 punch from 0x4da53be to 0x4da641d, (0x105f bytes) 2024-09-13T09:26:30.745 INFO:teuthology.orchestra.run.smithi063.stdout:1861 punch from 0x715f908 to 0x716d323, (0xda1b bytes) 2024-09-13T09:26:30.745 INFO:teuthology.orchestra.run.smithi063.stdout:1862 read 0x4d97ff3 thru 0x4d9a66a (0x2678 bytes) 2024-09-13T09:26:30.753 INFO:teuthology.orchestra.run.smithi063.stdout:1863 write 0xb440dc0 thru 0xb44e5ea (0xd82b bytes) 2024-09-13T09:26:30.766 INFO:teuthology.orchestra.run.smithi063.stdout:1864 read 0x4e19c06 thru 0x4e22a50 (0x8e4b bytes) 2024-09-13T09:26:30.767 INFO:teuthology.orchestra.run.smithi063.stdout:1865 read 0xab2f714 thru 0xab3ec53 (0xf540 bytes) 2024-09-13T09:26:30.768 INFO:teuthology.orchestra.run.smithi063.stdout:1866 read 0x8eb1fd7 thru 0x8ec0cbf (0xece9 bytes) 2024-09-13T09:26:30.768 INFO:teuthology.orchestra.run.smithi063.stdout:1867 punch from 0xafd0864 to 0xafdda96, (0xd232 bytes) 2024-09-13T09:26:30.768 INFO:teuthology.orchestra.run.smithi063.stdout:1869 write 0x122cf1 thru 0x12fc4e (0xcf5e bytes) 2024-09-13T09:26:30.773 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:30.772+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:30.777 INFO:teuthology.orchestra.run.smithi063.stdout:1870 trunc from 0xb44e5eb to 0xd749c9e 2024-09-13T09:26:30.787 INFO:teuthology.orchestra.run.smithi063.stdout:1871 read 0x2586b2a thru 0x2591011 (0xa4e8 bytes) 2024-09-13T09:26:30.788 INFO:teuthology.orchestra.run.smithi063.stdout:1872 write 0x845179a thru 0x845a596 (0x8dfd bytes) 2024-09-13T09:26:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:1873 trunc from 0xd749c9e to 0x5c1bee7 2024-09-13T09:26:30.832 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.833 INFO:teuthology.orchestra.run.smithi063.stdout:1875 punch from 0x1b45999 to 0x1b50015, (0xa67c bytes) 2024-09-13T09:26:30.836 INFO:teuthology.orchestra.run.smithi063.stdout:1876 trunc from 0x5c1bee7 to 0x8334c7a 2024-09-13T09:26:30.856 INFO:teuthology.orchestra.run.smithi063.stdout:1878 write 0x8a2063e thru 0x8a2e282 (0xdc45 bytes) 2024-09-13T09:26:30.860 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:30.856+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:30.864 INFO:teuthology.orchestra.run.smithi063.stdout:1879 write 0x1640c12 thru 0x1640e7c (0x26b bytes) 2024-09-13T09:26:30.867 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.867 INFO:teuthology.orchestra.run.smithi063.stdout:1882 read 0x1bc9aa thru 0x1c7ea0 (0xb4f7 bytes) 2024-09-13T09:26:30.869 INFO:teuthology.orchestra.run.smithi063.stdout:1883 write 0x645f667 thru 0x6467cb8 (0x8652 bytes) 2024-09-13T09:26:30.887 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.887 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:1884 writesame 0xeac8ab7 thru 0xead6a8a data_size 0x96(0xdfd4 bytes) 2024-09-13T09:26:30.902 INFO:teuthology.orchestra.run.smithi063.stdout:1885 read 0x1628b32 thru 0x1628f72 (0x441 bytes) 2024-09-13T09:26:30.903 INFO:teuthology.orchestra.run.smithi063.stdout:1886 punch from 0x3c29e30 to 0x3c3664d, (0xc81d bytes) 2024-09-13T09:26:30.903 INFO:teuthology.orchestra.run.smithi063.stdout:1887 write 0xb845a7 thru 0xb91f4f (0xd9a9 bytes) 2024-09-13T09:26:30.907 INFO:teuthology.orchestra.run.smithi063.stdout:1888 read 0x645a091 thru 0x6466c8d (0xcbfd bytes) 2024-09-13T09:26:30.908 INFO:teuthology.orchestra.run.smithi063.stdout:1889 write 0xa58a87 thru 0xa622fc (0x9876 bytes) 2024-09-13T09:26:30.912 INFO:teuthology.orchestra.run.smithi063.stdout:1891 write 0xe5cd13b thru 0xe5d53c2 (0x8288 bytes) 2024-09-13T09:26:30.918 INFO:teuthology.orchestra.run.smithi063.stdout:1893 punch from 0x6a1a760 to 0x6a1cc63, (0x2503 bytes) 2024-09-13T09:26:30.918 INFO:teuthology.orchestra.run.smithi063.stdout:1894 write 0x418cbb5 thru 0x4197d88 (0xb1d4 bytes) 2024-09-13T09:26:30.922 INFO:teuthology.orchestra.run.smithi063.stdout:1895 read 0x953a338 thru 0x954a232 (0xfefb bytes) 2024-09-13T09:26:30.922 INFO:teuthology.orchestra.run.smithi063.stdout:1896 punch from 0x26bd3be to 0x26ca329, (0xcf6b bytes) 2024-09-13T09:26:30.922 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:30.922 INFO:teuthology.orchestra.run.smithi063.stdout:1898 punch from 0x87bcc56 to 0x87c930f, (0xc6b9 bytes) 2024-09-13T09:26:30.922 INFO:teuthology.orchestra.run.smithi063.stdout:1901 read 0xbf63fe4 thru 0xbf7327b (0xf298 bytes) 2024-09-13T09:26:30.928 INFO:teuthology.orchestra.run.smithi063.stdout:1902 trunc from 0xead6a8b to 0xc21f247 2024-09-13T09:26:30.941 INFO:teuthology.orchestra.run.smithi063.stdout:1905 writesame 0x55c7572 thru 0x55ca315 data_size 0x2e(0x2da4 bytes) 2024-09-13T09:26:30.955 INFO:teuthology.orchestra.run.smithi063.stdout:1906 trunc from 0xc21f247 to 0x8d16c6d 2024-09-13T09:26:30.961 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:30.951+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:30.963 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.8.3 2024-09-13T09:26:30.975 INFO:teuthology.orchestra.run.smithi063.stdout:1907 read 0x6bf7073 thru 0x6bf7fc6 (0xf54 bytes) 2024-09-13T09:26:30.997 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:30.998 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:30.998 INFO:teuthology.orchestra.run.smithi063.stdout:1909 writesame 0xd33e18a thru 0xd3405cd data_size 0x16(0x2444 bytes) 2024-09-13T09:26:30.998 INFO:teuthology.orchestra.run.smithi063.stdout:1910 read 0x5f52917 thru 0x5f54b7f (0x2269 bytes) 2024-09-13T09:26:30.999 INFO:teuthology.orchestra.run.smithi063.stdout:1912 read 0xb12c56 thru 0xb1a58e (0x7939 bytes) 2024-09-13T09:26:31.000 INFO:teuthology.orchestra.run.smithi063.stdout:1913 punch from 0x5abdcd9 to 0x5acad34, (0xd05b bytes) 2024-09-13T09:26:31.000 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:31.000 INFO:teuthology.orchestra.run.smithi063.stdout:1915 read 0x215fbb4 thru 0x21629ae (0x2dfb bytes) 2024-09-13T09:26:31.001 INFO:teuthology.orchestra.run.smithi063.stdout:1916 write 0x319240f thru 0x3196215 (0x3e07 bytes) 2024-09-13T09:26:31.006 INFO:teuthology.orchestra.run.smithi063.stdout:1917 clone 16 order 25 su 16777216 sc 13 2024-09-13T09:26:31.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:31.062 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:31.055+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:31.110 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:31.110 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:31.120 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:31.164 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:31.160+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:31.237 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:31.237 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:31.249 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:31.249 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:31.264 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:31.255+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:31.361 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:31.361 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:31.365 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:31.359+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:31.375 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:31.392 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:31.417 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:31.436 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:31.458 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:31.467 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:31.459+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:31.481 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:26:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:31.505 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:31.510 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:31.510 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:31.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:31.510 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:31.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.557252411999997, 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:31.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:31.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:31.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:31.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:31.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:31.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.516 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 46.540694199000001, 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:31.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.521 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.305946941000002, 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:31.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:31.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:31.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:31.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.303737804000001, 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:31.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:31.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:31.532 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.299393279, 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:31.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.282842299999999, 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:31.537 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.282254291999998, 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:31.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:31.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.274548136, 2024-09-13T09:26:31.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:31.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:31.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:31.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.554 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:31.556 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:31.556 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:31.557 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:31.557 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:31.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:31.568 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:31.564+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:31.618 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone15 from 0xd3405ce (overlap 0x1cdf0ea) to 0x2e31b1 2024-09-13T09:26:31.669 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:31.663+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:31.718 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:31.718 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:31.718 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:31.718 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:31.729 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:31.766 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T09:26:31.770 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:31.763+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:31.781 INFO:teuthology.orchestra.run.smithi063.stdout:1918 write 0xbe8355 thru 0xbed323 (0x4fcf bytes) 2024-09-13T09:26:31.842 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:31.842 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:31.842 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:31.842 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:31.842 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:31.855 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:31.871 INFO:teuthology.orchestra.run.smithi063.stdout:1919 write 0x8224c6d thru 0x82333a2 (0xe736 bytes) 2024-09-13T09:26:31.872 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:31.863+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:31.875 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:31.875 INFO:teuthology.orchestra.run.smithi063.stdout:1921 punch from 0x28e9669 to 0x28ec0d7, (0x2a6e bytes) 2024-09-13T09:26:31.875 INFO:teuthology.orchestra.run.smithi063.stdout:1922 punch from 0x204557c to 0x2045abb, (0x53f bytes) 2024-09-13T09:26:31.883 INFO:teuthology.orchestra.run.smithi063.stdout:1924 trunc from 0xd3405ce to 0x8fe9cb1 2024-09-13T09:26:31.895 INFO:teuthology.orchestra.run.smithi063.stdout:1925 write 0x7c49dcc thru 0x7c5742f (0xd664 bytes) 2024-09-13T09:26:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:1926 write 0xd35b273 thru 0xd35b853 (0x5e1 bytes) 2024-09-13T09:26:31.923 INFO:teuthology.orchestra.run.smithi063.stdout:1927 read 0xd13e835 thru 0xd147d99 (0x9565 bytes) 2024-09-13T09:26:31.924 INFO:teuthology.orchestra.run.smithi063.stdout:1929 punch from 0x8d0e27d to 0x8d119ae, (0x3731 bytes) 2024-09-13T09:26:31.925 INFO:teuthology.orchestra.run.smithi063.stdout:1931 write 0x9fe3527 thru 0x9ff0d33 (0xd80d bytes) 2024-09-13T09:26:31.930 INFO:teuthology.orchestra.run.smithi063.stdout:1932 writesame 0x42f5246 thru 0x42fbae8 data_size 0x1(0x68a3 bytes) 2024-09-13T09:26:31.959 INFO:teuthology.orchestra.run.smithi063.stdout:1933 write 0xc66b7d6 thru 0xc673bd3 (0x83fe bytes) 2024-09-13T09:26:31.963 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:31.964 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:31.964 INFO:teuthology.orchestra.run.smithi063.stdout:1937 punch from 0xceb3360 to 0xceb3796, (0x436 bytes) 2024-09-13T09:26:31.964 INFO:teuthology.orchestra.run.smithi063.stdout:1939 writesame 0x8361426 thru 0x836ec11 data_size 0x1a(0xd7ec bytes) 2024-09-13T09:26:31.966 INFO:teuthology.orchestra.run.smithi063.stdout:1940 trunc from 0xd35b854 to 0xcf0b938 2024-09-13T09:26:31.971 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:31.963+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:31.977 INFO:teuthology.orchestra.run.smithi063.stdout:1941 write 0x95cc078 thru 0x95d2546 (0x64cf bytes) 2024-09-13T09:26:31.990 INFO:teuthology.orchestra.run.smithi063.stdout:1942 trunc from 0xcf0b938 to 0x69f27e7 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.033129434999999, 2024-09-13T09:26:31.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:31.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:26:31.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:26:31.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:26:31.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:31.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4605 3.19 3:9a4feb2f:::rbd_data.11189fb5ddd6.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.987877+0000", 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.999082152, 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022921418999999998, 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4605 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987877+0000", 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987877+0000", 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987875+0000", 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987882+0000", 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987895+0000", 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3376e-05 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987902+0000", 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.987946+0000", 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3930999999999997e-05 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.988031+0000", 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5483999999999997e-05 2024-09-13T09:26:32.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.991294+0000", 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0032626249999999999 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:32.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010644+0000", 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019350046999999999 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010694+0000", 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9910000000000002e-05 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010747+0000", 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2964999999999999e-05 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.010798+0000", 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1146999999999997e-05 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:27650 3.0 3:02c7c676:::rbd_data.111811db2be7.0000000000000007:f [read 1003520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.317800+0000", 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6691586469999997, 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01316146, 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.006 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 27650 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.317800+0000", 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.317800+0000", 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.317799+0000", 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.317803+0000", 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.546e-06 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.317810+0000", 2024-09-13T09:26:32.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4410000000000004e-06 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.317816+0000", 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0900000000000001e-06 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.327516+0000", 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096994290000000007 2024-09-13T09:26:32.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.327610+0000", 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4111999999999996e-05 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.330961+0000", 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033515020000000001 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.010 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28080 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 733184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.328902+0000", 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6580564330000005, 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010237313, 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28080 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328902+0000", 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328902+0000", 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328902+0000", 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328904+0000", 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7949999999999999e-06 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328911+0000", 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4110000000000001e-06 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328917+0000", 2024-09-13T09:26:32.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3589999999999996e-06 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339011+0000", 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010094089000000001 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339076+0000", 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4321999999999998e-05 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339140+0000", 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3953000000000002e-05 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28081 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 761856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.328921+0000", 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6580374620000002, 2024-09-13T09:26:32.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010413525999999999, 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28081 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328921+0000", 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328921+0000", 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328921+0000", 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328923+0000", 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:26:32.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328929+0000", 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.609e-06 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.328934+0000", 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7960000000000003e-06 2024-09-13T09:26:32.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339165+0000", 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010230998999999999 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339247+0000", 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2379000000000005e-05 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339335+0000", 2024-09-13T09:26:32.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7797e-05 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28115 3.6 3:64cb1aaf:::rbd_data.111811db2be7.0000000000000008:f [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329227+0000", 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.657731611, 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010364043, 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.020 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28115 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329227+0000", 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329227+0000", 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329227+0000", 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329236+0000", 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0999999999999993e-06 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329242+0000", 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0889999999999996e-06 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329247+0000", 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339419+0000", 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01017241 2024-09-13T09:26:32.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339501+0000", 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1526999999999994e-05 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339591+0000", 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0251e-05 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.024 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28116 3.6 3:64cb1aaf:::rbd_data.111811db2be7.0000000000000008:f [read 208896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329251+0000", 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6577074849999995, 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010516694, 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28116 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329251+0000", 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329251+0000", 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329251+0000", 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329253+0000", 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5040000000000001e-06 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329258+0000", 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0869999999999997e-06 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329263+0000", 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339617+0000", 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010354194000000001 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339684+0000", 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7166e-05 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339768+0000", 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3924999999999994e-05 2024-09-13T09:26:32.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28117 3.6 3:64cb1aaf:::rbd_data.111811db2be7.0000000000000008:f [read 237568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329267+0000", 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.657692129, 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010699525, 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28117 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329267+0000", 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329267+0000", 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329266+0000", 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329269+0000", 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3209999999999999e-06 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329274+0000", 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5820000000000001e-06 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329279+0000", 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339793+0000", 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010514425000000001 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339865+0000", 2024-09-13T09:26:32.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1599000000000003e-05 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339966+0000", 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010121200000000001 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28118 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329283+0000", 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6576759469999995, 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010228391999999999, 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28118 2024-09-13T09:26:32.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329283+0000", 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329283+0000", 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329282+0000", 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329285+0000", 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.26e-06 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329293+0000", 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7910000000000005e-06 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329298+0000", 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9630000000000003e-06 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339350+0000", 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01005172 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339414+0000", 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4188000000000003e-05 2024-09-13T09:26:32.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339511+0000", 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6745999999999998e-05 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28119 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329303+0000", 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6576560709999999, 2024-09-13T09:26:32.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01033543, 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28119 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329303+0000", 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329303+0000", 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329302+0000", 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329305+0000", 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.255e-06 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329310+0000", 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4550000000000003e-06 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329314+0000", 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3409999999999996e-06 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339540+0000", 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010226035 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339593+0000", 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2960000000000001e-05 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339638+0000", 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4697999999999998e-05 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28120 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 331776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329343+0000", 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6576152200000003, 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01039868, 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.043 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28120 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329343+0000", 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329343+0000", 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329343+0000", 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329346+0000", 2024-09-13T09:26:32.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4089999999999999e-06 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329353+0000", 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4620000000000001e-06 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329358+0000", 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2469999999999999e-06 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339653+0000", 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010295327999999999 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339697+0000", 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3989000000000002e-05 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339742+0000", 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4845999999999997e-05 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.047 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28121 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329362+0000", 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6575967900000004, 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010470791, 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28121 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329362+0000", 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329362+0000", 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329361+0000", 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329364+0000", 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.509e-06 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329369+0000", 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1869999999999996e-06 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329377+0000", 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4830000000000002e-06 2024-09-13T09:26:32.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339758+0000", 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010381054000000001 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339790+0000", 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2706999999999997e-05 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339833+0000", 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2348999999999997e-05 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28122 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 389120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329381+0000", 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6575781349999996, 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01060182, 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28122 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329381+0000", 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329381+0000", 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329380+0000", 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329383+0000", 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6460000000000002e-06 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329388+0000", 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4349999999999996e-06 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329393+0000", 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339846+0000", 2024-09-13T09:26:32.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010452997 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339883+0000", 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6627e-05 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.339982+0000", 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9747000000000006e-05 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28123 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 446464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329397+0000", 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6575618990000001, 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010850185, 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28123 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329397+0000", 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329397+0000", 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329396+0000", 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329399+0000", 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.39e-06 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329404+0000", 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0509999999999996e-06 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329409+0000", 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7280000000000003e-06 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340117+0000", 2024-09-13T09:26:32.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010708495 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340182+0000", 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4449000000000006e-05 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340247+0000", 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5382000000000005e-05 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28124 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 417792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329413+0000", 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.657545689, 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011040937000000001, 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28124 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329413+0000", 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329413+0000", 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329413+0000", 2024-09-13T09:26:32.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329415+0000", 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329420+0000", 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4280000000000004e-06 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329425+0000", 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4619999999999998e-06 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340283+0000", 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010858322 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340358+0000", 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4462000000000006e-05 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340454+0000", 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6280000000000007e-05 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28125 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329429+0000", 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.657529802, 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011176704000000001, 2024-09-13T09:26:32.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28125 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329429+0000", 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329429+0000", 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329429+0000", 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329431+0000", 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.249e-06 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329436+0000", 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3009999999999997e-06 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329441+0000", 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6550000000000003e-06 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340491+0000", 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011050446 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340544+0000", 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3239000000000003e-05 2024-09-13T09:26:32.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340606+0000", 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1146e-05 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:28126 3.4 3:2674a6d8:::rbd_data.111811db2be7.000000000000000d:f [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.329446+0000", 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6575125719999999, 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011338566, 2024-09-13T09:26:32.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 28126 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329446+0000", 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329446+0000", 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329446+0000", 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329448+0000", 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:26:32.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329455+0000", 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8009999999999998e-06 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.329460+0000", 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4979999999999997e-06 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340629+0000", 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011168356000000001 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340700+0000", 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1353999999999995e-05 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.340785+0000", 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4528000000000001e-05 2024-09-13T09:26:32.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:31241 3.6 3:661c5e85:::rbd_object_map.1118f0c8e51e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:28.801510+0000", 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.1854487109999998, 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011254544, 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 31241 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.801510+0000", 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.801510+0000", 2024-09-13T09:26:32.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.801509+0000", 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.801517+0000", 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.824e-06 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.801530+0000", 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3604999999999999e-05 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.801539+0000", 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6969999999999999e-06 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.808154+0000", 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066150860000000001 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.808285+0000", 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013091099999999999 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.808903+0000", 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00061805400000000004 2024-09-13T09:26:32.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.809991+0000", 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010876410000000001 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.812601+0000", 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002610224 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.812702+0000", 2024-09-13T09:26:32.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010111300000000001 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:28.812765+0000", 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2384000000000004e-05 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:35690 3.19 3:9d20c047:::rbd_data.11185dd0f6b.0000000000000006:head [stat,truncate 245760] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.651441+0000", 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.33551787, 2024-09-13T09:26:32.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01146954, 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 35690 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651441+0000", 2024-09-13T09:26:32.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651441+0000", 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651439+0000", 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651451+0000", 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2193e-05 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651511+0000", 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0322000000000003e-05 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651526+0000", 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4893e-05 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651643+0000", 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011691700000000001 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651806+0000", 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016294399999999999 2024-09-13T09:26:32.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.662596+0000", 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010790127 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.662714+0000", 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117742 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.662910+0000", 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019642499999999999 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:35694 3.18 3:1c316563:::rbd_data.11185dd0f6b.000000000000000a:head [stat,truncate 245760] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.651660+0000", 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.33529889200000002, 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010272969999999999, 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:32.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 35694 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651660+0000", 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651660+0000", 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651658+0000", 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651668+0000", 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.011e-05 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651684+0000", 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:26:32.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651694+0000", 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0426e-05 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.651784+0000", 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9759000000000006e-05 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.652006+0000", 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022158599999999999 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.661767+0000", 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0097614469999999995 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.661834+0000", 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7161000000000003e-05 2024-09-13T09:26:32.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.661933+0000", 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8646000000000004e-05 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.094 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:32.096 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:32.097 INFO:teuthology.orchestra.run.smithi063.stdout:1944 trunc from 0x69f27e7 to 0x2c40fcd 2024-09-13T09:26:32.097 INFO:teuthology.orchestra.run.smithi063.stdout:1946 punch from 0x28f7b14 to 0x290060a, (0x8af6 bytes) 2024-09-13T09:26:32.097 INFO:teuthology.orchestra.run.smithi063.stdout:1947 flatten 2024-09-13T09:26:32.097 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:32.092+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:32.111 INFO:teuthology.orchestra.run.smithi063.stdout:1951 trunc from 0x2c40fcd to 0xb7b4343 2024-09-13T09:26:32.124 INFO:teuthology.orchestra.run.smithi063.stdout:1952 read 0x6481772 thru 0x6481bbb (0x44a bytes) 2024-09-13T09:26:32.140 INFO:teuthology.orchestra.run.smithi063.stdout:1953 writesame 0xe07a7b5 thru 0xe0894b0 data_size 0x3a(0xecfc bytes) 2024-09-13T09:26:32.143 INFO:teuthology.orchestra.run.smithi063.stdout:1954 read 0x5f5e40f thru 0x5f68670 (0xa262 bytes) 2024-09-13T09:26:32.143 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:32.144 INFO:teuthology.orchestra.run.smithi063.stdout:1956 trunc from 0xe0894b1 to 0xd5712b0 2024-09-13T09:26:32.160 INFO:teuthology.orchestra.run.smithi063.stdout:1959 trunc from 0xd5712b0 to 0xb8f1ad4 2024-09-13T09:26:32.173 INFO:teuthology.orchestra.run.smithi063.stdout:1960 punch from 0xb307c9f to 0xb316878, (0xebd9 bytes) 2024-09-13T09:26:32.173 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:32.173 INFO:teuthology.orchestra.run.smithi063.stdout:1962 punch from 0xb222ea7 to 0xb22ce4d, (0x9fa6 bytes) 2024-09-13T09:26:32.173 INFO:teuthology.orchestra.run.smithi063.stdout:1963 punch from 0x2908809 to 0x29142f6, (0xbaed bytes) 2024-09-13T09:26:32.175 INFO:teuthology.orchestra.run.smithi063.stdout:get_pg_num_history pg_num_history pg_num_history(e46 pg_nums {1={16=8},2={17=1},3={21=32}} deleted_pools ) 2024-09-13T09:26:32.175 INFO:teuthology.orchestra.run.smithi063.stdout:Importing pgid 3.8 2024-09-13T09:26:32.176 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:111884c9:::rbd_data.1118e3e5e25d.00000000000000bf:head# 2024-09-13T09:26:32.176 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.176 INFO:teuthology.orchestra.run.smithi063.stdout:1966 trunc from 0xb8f1ad4 to 0xdadf32d 2024-09-13T09:26:32.178 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:121fe3f4:::rbd_data.1118a28cd5c4.0000000000000007:head# 2024-09-13T09:26:32.178 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.181 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:1269e70a:::rbd_data.1118a28cd5c4.0000000000000009:head# 2024-09-13T09:26:32.181 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.183 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:127fce81:::rbd_data.111851704b2b.0000000000000001:9# 2024-09-13T09:26:32.186 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:127fce81:::rbd_data.111851704b2b.0000000000000001:head# 2024-09-13T09:26:32.186 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 9=[]:{9=[9]} 2024-09-13T09:26:32.186 INFO:teuthology.orchestra.run.smithi063.stdout:1967 write 0x5d8093e thru 0x5d88a49 (0x810c bytes) 2024-09-13T09:26:32.187 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:13ad7219:::rbd_data.1118a28cd5c4.0000000000000032:head# 2024-09-13T09:26:32.187 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.189 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:1480b51f:::rbd_data.1118a28cd5c4.0000000000000054:head# 2024-09-13T09:26:32.190 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.192 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:15cb164d:::rbd_data.1118a28cd5c4.00000000000000d3:head# 2024-09-13T09:26:32.193 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.194 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:16185ba7:::rbd_data.111867511a81.0000000000000009:5# 2024-09-13T09:26:32.196 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:16185ba7:::rbd_data.111867511a81.0000000000000009:head# 2024-09-13T09:26:32.196 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 5=[]:{5=[5]} 2024-09-13T09:26:32.196 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:32.187+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:32.197 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:163352f1:::rbd_object_map.11189888a71f:head# 2024-09-13T09:26:32.197 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.198 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:1718b63c:::rbd_data.1118e3e5e25d.00000000000000bc:head# 2024-09-13T09:26:32.198 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.199 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:173d6c83:::rbd_data.1118e98b86fc.0000000000000013:d# 2024-09-13T09:26:32.201 INFO:teuthology.orchestra.run.smithi063.stdout:1969 write 0xe22f5a0 thru 0xe230f3c (0x199d bytes) 2024-09-13T09:26:32.202 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:173d6c83:::rbd_data.1118e98b86fc.0000000000000013:head# 2024-09-13T09:26:32.202 INFO:teuthology.orchestra.run.smithi063.stdout:snapset d=[]:{d=[d]} 2024-09-13T09:26:32.203 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:1752c46c:::rbd_data.1118a28cd5c4.0000000000000000:head# 2024-09-13T09:26:32.203 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.204 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:32.205 INFO:teuthology.orchestra.run.smithi063.stdout:1971 write 0x6a61705 thru 0x6a64f45 (0x3841 bytes) 2024-09-13T09:26:32.205 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:17e81ee6:::rbd_id.image_client.0-clone4:head# 2024-09-13T09:26:32.205 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:26:32.206 INFO:teuthology.orchestra.run.smithi063.stdout:write_pg epoch 46 info 3.8( v 46'129 (0'0,46'129] local-lis/les=21/22 n=16 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-13T09:26:32.208 INFO:teuthology.orchestra.run.smithi063.stdout:Import successful 2024-09-13T09:26:32.211 INFO:teuthology.orchestra.run.smithi063.stdout:1972 punch from 0x895d5db to 0x8960cc3, (0x36e8 bytes) 2024-09-13T09:26:32.211 INFO:teuthology.orchestra.run.smithi063.stdout:1974 write 0x73bd8af thru 0x73c3ba6 (0x62f8 bytes) 2024-09-13T09:26:32.216 INFO:teuthology.orchestra.run.smithi063.stdout:1976 read 0x338772f thru 0x338c2a4 (0x4b76 bytes) 2024-09-13T09:26:32.216 INFO:teuthology.orchestra.run.smithi063.stdout:1977 write 0x1081941 thru 0x108ab08 (0x91c8 bytes) 2024-09-13T09:26:32.219 INFO:teuthology.orchestra.run.smithi063.stdout:1978 read 0xd1d1884 thru 0xd1d5455 (0x3bd2 bytes) 2024-09-13T09:26:32.220 INFO:teuthology.orchestra.run.smithi063.stdout:1979 punch from 0xa273d51 to 0xa27dbf6, (0x9ea5 bytes) 2024-09-13T09:26:32.229 INFO:teuthology.orchestra.run.smithi063.stdout:1980 trunc from 0xe230f3d to 0x8368168 2024-09-13T09:26:32.238 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:32.238 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:32.238 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:32.238 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:32.243 INFO:teuthology.orchestra.run.smithi063.stdout:1981 write 0x52edbcb thru 0x52f1e57 (0x428d bytes) 2024-09-13T09:26:32.247 INFO:teuthology.orchestra.run.smithi063.stdout:1982 punch from 0x1ef949f to 0x1f08926, (0xf487 bytes) 2024-09-13T09:26:32.249 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:32.256 INFO:teuthology.orchestra.run.smithi063.stdout:1983 trunc from 0x8368168 to 0xd891fec 2024-09-13T09:26:32.268 INFO:teuthology.orchestra.run.smithi063.stdout:1984 read 0x5d56593 thru 0x5d607b5 (0xa223 bytes) 2024-09-13T09:26:32.269 INFO:teuthology.orchestra.run.smithi063.stdout:1985 read 0x80bba1f thru 0x80c48d5 (0x8eb7 bytes) 2024-09-13T09:26:32.269 INFO:teuthology.orchestra.run.smithi063.stdout:1986 writesame 0xcce126a thru 0xcce7ddb data_size 0x11(0x6b72 bytes) 2024-09-13T09:26:32.297 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:32.291+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:32.299 INFO:teuthology.orchestra.run.smithi063.stdout:1987 trunc from 0xd891fec to 0x6f37bd 2024-09-13T09:26:32.336 INFO:teuthology.orchestra.run.smithi063.stdout:1988 trunc from 0x6f37bd to 0x56465a0 2024-09-13T09:26:32.346 INFO:teuthology.orchestra.run.smithi063.stdout:1989 write 0x1b6804b thru 0x1b7768d (0xf643 bytes) 2024-09-13T09:26:32.364 INFO:teuthology.orchestra.run.smithi063.stdout:1991 writesame 0x6767cf7 thru 0x676849c data_size 0x16(0x7a6 bytes) 2024-09-13T09:26:32.364 INFO:teuthology.orchestra.run.smithi063.stdout:1992 writesame 0x3e39399 thru 0x3e3cd34 data_size 0x6(0x399c bytes) 2024-09-13T09:26:32.365 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:32.365 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:32.365 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:32.365 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:32.365 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:32.376 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:32.385 INFO:teuthology.orchestra.run.smithi063.stdout:1993 write 0x7b500a7 thru 0x7b59f16 (0x9e70 bytes) 2024-09-13T09:26:32.407 INFO:teuthology.orchestra.run.smithi063.stdout:1994 write 0x9a4df5d thru 0x9a501e5 (0x2289 bytes) 2024-09-13T09:26:32.407 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:32.399+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:32.413 INFO:teuthology.orchestra.run.smithi063.stdout:1995 read 0x75fe27 thru 0x76af3a (0xb114 bytes) 2024-09-13T09:26:32.430 INFO:teuthology.orchestra.run.smithi063.stdout:1997 writesame 0xc23f033 thru 0xc248376 data_size 0x34(0x9344 bytes) 2024-09-13T09:26:32.431 INFO:teuthology.orchestra.run.smithi063.stdout:1998 punch from 0x31d2c82 to 0x31d663e, (0x39bc bytes) 2024-09-13T09:26:32.431 INFO:teuthology.orchestra.run.smithi063.stdout:1999 writesame 0xb02d0b thru 0xb0f557 data_size 0x1(0xc84d bytes) 2024-09-13T09:26:32.471 INFO:teuthology.orchestra.run.smithi063.stdout:2000 punch from 0x5f4a16f to 0x5f52611, (0x84a2 bytes) 2024-09-13T09:26:32.487 INFO:teuthology.orchestra.run.smithi063.stdout:2002 trunc from 0xc248377 to 0x46201d3 2024-09-13T09:26:32.496 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:32.496 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:276 3.4 e27/21 3:27927849:::rbd_data.111867511a81.0000000000000026:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.987758741999997, 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002929961, 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504324+0000", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504366+0000", 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2412e-05 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504407+0000", 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0329e-05 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504427+0000", 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0089999999999999e-05 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504518+0000", 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0801e-05 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504566+0000", 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8745999999999997e-05 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507192+0000", 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00262546 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507256+0000", 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:307 3.9 e27/21 3:92c4c1be:::rbd_data.111867511a81.0000000000000087:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.862512270000003, 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002303486, 2024-09-13T09:26:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629572+0000", 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629570+0000", 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629585+0000", 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4826e-05 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629615+0000", 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9879e-05 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629628+0000", 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2673e-05 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629749+0000", 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121304 2024-09-13T09:26:32.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.629816+0000", 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7285000000000004e-05 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.631763+0000", 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019471040000000001 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.631876+0000", 2024-09-13T09:26:32.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112151 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.689145746999998, 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:26:32.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:26:32.510 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.280969861999999, 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:26:32.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.280779739, 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.515 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:26:32.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:26:32.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.279967330000002, 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.520 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:26:32.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.091060680999998, 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.525 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:26:32.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.529 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4109 3.1f e35/25 3:fb0b19d8:::rbd_data.1118fb4e889d.0000000000000006:head v 35'26, mlcod=31'23)", 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.025989911, 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028372279999999998, 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466099+0000", 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7079999999999996e-06 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466120+0000", 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1491000000000001e-05 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466130+0000", 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466228+0000", 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8072000000000006e-05 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466258+0000", 2024-09-13T09:26:32.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9842e-05 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.468885+0000", 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026277119999999999 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.468932+0000", 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6322000000000001e-05 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.525951295999999, 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.452412283000001, 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.539 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:26:32.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.359868915, 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:32.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:6576 3.1f e38/25 3:fff74333:::rbd_data.1118e98b86fc.0000000000000003:head v 38'31, mlcod=38'29)", 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:14.090448+0000", 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.401636630999999, 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002258323, 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090448+0000", 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090448+0000", 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090446+0000", 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090812+0000", 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036630899999999999 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090849+0000", 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7054000000000001e-05 2024-09-13T09:26:32.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090863+0000", 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3583e-05 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090952+0000", 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9294000000000003e-05 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.090982+0000", 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9663e-05 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.092630+0000", 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016485440000000001 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:14.092706+0000", 2024-09-13T09:26:32.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5869000000000002e-05 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:24464 3.9 e45/21 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:head v 45'68, mlcod=44'66)", 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3402731879999994, 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028365740000000001, 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151810+0000", 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152495+0000", 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00068434500000000005 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152521+0000", 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6288e-05 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152534+0000", 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:26:32.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152635+0000", 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100758 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152669+0000", 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4826999999999997e-05 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.154591+0000", 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001921407 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.154648+0000", 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6826000000000001e-05 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:32.558 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.3318083390000002, 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:26:32.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:26:32.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.3175505560000005, 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.563 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:26:32.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:27505 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'70, mlcod=45'68)", 2024-09-13T09:26:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.312823+0000", 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.1792608789999992, 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023505010000000001, 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.312823+0000", 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.312823+0000", 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.312822+0000", 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.312837+0000", 2024-09-13T09:26:32.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4640999999999999e-05 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.312858+0000", 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1157000000000001e-05 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.312867+0000", 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6279999999999994e-06 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.313025+0000", 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015879800000000001 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.313043+0000", 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7365000000000001e-05 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.315128+0000", 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020854229999999999 2024-09-13T09:26:32.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.315174+0000", 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5804999999999999e-05 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:30766 3.9 e49/21 3:976c9d51:::rbd_data.111812ca204d.00000000000000b4:head v 49'85, mlcod=49'83)", 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:27.486343+0000", 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0057409740000001, 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002378781, 2024-09-13T09:26:32.572 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486343+0000", 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486343+0000", 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486341+0000", 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486369+0000", 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7412000000000001e-05 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486414+0000", 2024-09-13T09:26:32.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4962000000000001e-05 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486433+0000", 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9086e-05 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486519+0000", 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6422999999999996e-05 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.486573+0000", 2024-09-13T09:26:32.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3499999999999999e-05 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.488655+0000", 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020822200000000001 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:27.488722+0000", 2024-09-13T09:26:32.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7335999999999994e-05 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35667 3.0 e52/21 3:07bd3fb7:::rbd_data.11185dd0f6b.0000000000000020:head v 52'123, mlcod=51'120)", 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.625573+0000", 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.86651113599999996, 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024456790000000001, 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625573+0000", 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625573+0000", 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625572+0000", 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:32.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625586+0000", 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4744e-05 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625617+0000", 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0071000000000001e-05 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625631+0000", 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4912000000000001e-05 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625760+0000", 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012875199999999999 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625835+0000", 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5018000000000006e-05 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.627958+0000", 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021224590000000002 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.628019+0000", 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1138999999999999e-05 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.581 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.848832103, 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:26:32.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:26:32.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:36028 3.4 e52/21 3:24d6dfd7:::rbd_data.11184f2468.0000000000000006:head v 52'211, mlcod=52'209)", 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:32.206540+0000", 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.28554411200000002, 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022751049999999999, 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.587 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206540+0000", 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206540+0000", 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206538+0000", 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206586+0000", 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7534000000000003e-05 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206616+0000", 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0204000000000001e-05 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206629+0000", 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2621e-05 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206889+0000", 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026064300000000002 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.206933+0000", 2024-09-13T09:26:32.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3322999999999998e-05 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.208738+0000", 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001805751 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:32.208815+0000", 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6825999999999999e-05 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.591 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:32.592 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:32.592 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:32.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:32.594 INFO:teuthology.orchestra.run.smithi063.stdout:2003 punch from 0x2ace0f3 to 0x2ad014c, (0x2059 bytes) 2024-09-13T09:26:32.594 INFO:teuthology.orchestra.run.smithi063.stdout:2004 trunc from 0x46201d3 to 0x343e43 2024-09-13T09:26:32.594 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:32.594 INFO:teuthology.orchestra.run.smithi063.stdout:2007 writesame 0x8e493b0 thru 0x8e4d8d1 data_size 0x1(0x4522 bytes) 2024-09-13T09:26:32.594 INFO:teuthology.orchestra.run.smithi063.stdout:2008 clone 17 order 22 su 16384 sc 3 2024-09-13T09:26:32.595 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:32.592+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:32.694 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:32.692+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:32.711 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:32.711 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:32.711 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:32.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:32.771 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /var/log/ceph/exp.3.8.3 2024-09-13T09:26:32.795 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:32.792+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:32.830 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:32.830 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:32.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:32.895 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:32.892+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3063 3.9 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.606971239, 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031119964999999999, 2024-09-13T09:26:32.993 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3063 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382211+0000", 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382223+0000", 2024-09-13T09:26:32.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2245e-05 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382253+0000", 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9947999999999999e-05 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382267+0000", 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4034e-05 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382338+0000", 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0931000000000002e-05 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382439+0000", 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001009 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.390414+0000", 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079749260000000002 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413261+0000", 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022847375 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:32.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413269+0000", 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413302+0000", 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3262000000000001e-05 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413333+0000", 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0077999999999999e-05 2024-09-13T09:26:32.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22720 3.1b 3:d9bd9d03:::rbd_data.111811db2be7.00000000000000b7:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.759507599999999, 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033396362999999998, 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22720 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229675+0000", 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229684+0000", 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229703+0000", 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9276999999999999e-05 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229713+0000", 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6979999999999993e-06 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229770+0000", 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7096999999999999e-05 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229856+0000", 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5376000000000004e-05 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.236948+0000", 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070919360000000001 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262783+0000", 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025835137000000001 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262895+0000", 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011265 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262993+0000", 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8052000000000002e-05 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.263073+0000", 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9244000000000002e-05 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25507 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5906432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954507+0000", 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0346767210000003, 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029582375000000001, 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25507 2024-09-13T09:26:33.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954507+0000", 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954507+0000", 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954506+0000", 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954511+0000", 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5569999999999997e-06 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954520+0000", 2024-09-13T09:26:33.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3230000000000004e-06 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954528+0000", 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0080000000000002e-06 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.983985+0000", 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029456782000000001 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984028+0000", 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3278999999999997e-05 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984089+0000", 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1422999999999998e-05 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25508 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5935104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0346492489999992, 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029767913999999999, 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25508 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:33.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954534+0000", 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954538+0000", 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8010000000000001e-06 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954546+0000", 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5499999999999995e-06 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954553+0000", 2024-09-13T09:26:33.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0049999999999998e-06 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984129+0000", 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029576203999999998 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984202+0000", 2024-09-13T09:26:33.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000006e-05 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984303+0000", 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026600000000001 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25509 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5963776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0346247870000003, 2024-09-13T09:26:33.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029860339, 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25509 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954562+0000", 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3100000000000001e-06 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954571+0000", 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.816e-06 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954578+0000", 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984320+0000", 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029742527000000001 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984361+0000", 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0615999999999999e-05 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984419+0000", 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8594000000000002e-05 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25510 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5992448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0346000659999994, 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030038194000000001, 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25510 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954583+0000", 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954587+0000", 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5640000000000001e-06 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954595+0000", 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.473e-06 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954602+0000", 2024-09-13T09:26:33.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984458+0000", 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029856186 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984529+0000", 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0345000000000005e-05 2024-09-13T09:26:33.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984622+0000", 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3197999999999998e-05 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25511 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6021120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0345753959999993, 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030144265, 2024-09-13T09:26:33.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25511 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954632+0000", 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3645000000000001e-05 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954641+0000", 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4490000000000005e-06 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954649+0000", 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3750000000000003e-06 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984638+0000", 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029988471999999999 2024-09-13T09:26:33.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984678+0000", 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0170999999999999e-05 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984753+0000", 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4666999999999995e-05 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25512 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6033408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0345277540000009, 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030307508, 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25512 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954655+0000", 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954659+0000", 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954668+0000", 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954677+0000", 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1530000000000003e-06 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984792+0000", 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030114499999999999 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984866+0000", 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4437000000000006e-05 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984964+0000", 2024-09-13T09:26:33.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7330000000000005e-05 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25513 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6062080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0345022149999998, 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030400765999999999, 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25513 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954681+0000", 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954687+0000", 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5039999999999999e-06 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954693+0000", 2024-09-13T09:26:33.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0499999999999999e-06 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954700+0000", 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3089999999999998e-06 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984980+0000", 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280509000000001 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985022+0000", 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1301000000000003e-05 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985082+0000", 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0726000000000001e-05 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25514 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6090752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:33.037 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.034477764, 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030608730000000001, 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25514 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954705+0000", 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954709+0000", 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7290000000000002e-06 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954718+0000", 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8049999999999996e-06 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954725+0000", 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985121+0000", 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030396636000000001 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985193+0000", 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1874999999999999e-05 2024-09-13T09:26:33.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985315+0000", 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121452 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25515 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6119424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0344526980000008, 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030701790999999999, 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25515 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954730+0000", 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954734+0000", 2024-09-13T09:26:33.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8449999999999996e-06 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954744+0000", 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0222e-05 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954752+0000", 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2049999999999996e-06 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985334+0000", 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030582024999999999 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985375+0000", 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1253e-05 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985433+0000", 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7986999999999999e-05 2024-09-13T09:26:33.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25516 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6148096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.034426281, 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030869497999999999, 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25516 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954757+0000", 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954761+0000", 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7479999999999999e-06 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954803+0000", 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2342000000000002e-05 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954811+0000", 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985473+0000", 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030661702999999998 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985541+0000", 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7800999999999997e-05 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985627+0000", 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6446000000000005e-05 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25517 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6176768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0343648969999997, 2024-09-13T09:26:33.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030924971999999998, 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25517 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954818+0000", 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954823+0000", 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954834+0000", 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1174e-05 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954842+0000", 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0339999999999997e-06 2024-09-13T09:26:33.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985645+0000", 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030803298 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985687+0000", 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2089000000000003e-05 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985744+0000", 2024-09-13T09:26:33.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6666999999999998e-05 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25518 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6205440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0343352299999999, 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031093369999999999, 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25518 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:33.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954848+0000", 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954852+0000", 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8820000000000003e-06 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954860+0000", 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954868+0000", 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6680000000000001e-06 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985784+0000", 2024-09-13T09:26:33.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03091648 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985852+0000", 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7407e-05 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985942+0000", 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25519 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6234112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0343089360000004, 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031185332, 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25519 2024-09-13T09:26:33.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954874+0000", 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954878+0000", 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954886+0000", 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6209999999999993e-06 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954894+0000", 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3409999999999997e-06 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985960+0000", 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031066527 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986002+0000", 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1925999999999998e-05 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986060+0000", 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7948999999999997e-05 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25520 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6262784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.034281, 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031383412999999999, 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.065 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25520 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954902+0000", 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954906+0000", 2024-09-13T09:26:33.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8060000000000001e-06 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954914+0000", 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3059999999999998e-06 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954923+0000", 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7579999999999998e-06 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986106+0000", 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031182399 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986184+0000", 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8699e-05 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986286+0000", 2024-09-13T09:26:33.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101977 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25563 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0342367810000006, 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034890003000000003, 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25563 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954946+0000", 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954953+0000", 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954963+0000", 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0468e-05 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954971+0000", 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2600000000000005e-06 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986090+0000", 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031118930999999999 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986167+0000", 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6470999999999993e-05 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986448+0000", 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000281451 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989625+0000", 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003177352 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989740+0000", 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114632 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989798+0000", 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7555e-05 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989837+0000", 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9385999999999999e-05 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25577 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.075 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0293154910000002, 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032046094999999997, 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25577 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959867+0000", 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959875+0000", 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9859999999999994e-06 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959888+0000", 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2977e-05 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959898+0000", 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9280000000000005e-06 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986894+0000", 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026995473999999998 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.987025+0000", 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013186200000000001 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.987422+0000", 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039672500000000001 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.990604+0000", 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00318207 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991786+0000", 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001181229 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991869+0000", 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3726000000000004e-05 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991914+0000", 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207000000000003e-05 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:35809 3.3 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 12485461~20431] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:31.818266+0000", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1709173740000001, 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051355610000000003, 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35809 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.818266+0000", 2024-09-13T09:26:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.818266+0000", 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.818265+0000", 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.835941+0000", 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017675535999999999 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.835978+0000", 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6888000000000003e-05 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.835990+0000", 2024-09-13T09:26:33.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960999999999999e-05 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.836072+0000", 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2557000000000003e-05 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.836272+0000", 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000199784 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.837836+0000", 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001564223 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.863431+0000", 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025595374000000001 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869497+0000", 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060658170000000003 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869570+0000", 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2679e-05 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869622+0000", 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2113000000000001e-05 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:36139 3.3 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:32.431022+0000", 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.55816212700000001, 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039760744000000001, 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 36139 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431022+0000", 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431022+0000", 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431020+0000", 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:33.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431032+0000", 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.255e-05 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431065+0000", 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3154999999999997e-05 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431080+0000", 2024-09-13T09:26:33.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4714e-05 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431154+0000", 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4158000000000004e-05 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431283+0000", 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012883999999999999 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:33.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.440863+0000", 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095793730000000004 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470663+0000", 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029800462999999999 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470705+0000", 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2352999999999999e-05 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470744+0000", 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8495000000000003e-05 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470782+0000", 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8618999999999997e-05 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.094 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:33.095 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:33.095 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:33.092+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:33.197 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:33.187+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:33.255 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:33.255 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:33.255 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:33.265 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:33.297 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:33.291+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:33.372 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:33.373 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:33.373 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:33.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:33.398 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:33.396+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:33.491 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.024538954000001, 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.000196691999999, 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.995828328999998, 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.632399671000002, 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.79608919, 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.543083161999999, 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:33.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:33.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.526 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29819 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0388460239999997, 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022753847000000001, 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29819 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450415+0000", 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450418+0000", 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450423+0000", 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450428+0000", 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463252+0000", 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012823467999999999 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463451+0000", 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019896599999999999 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473170+0000", 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009718829 2024-09-13T09:26:33.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29887 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0385983670000005, 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015606436, 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29887 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:33.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450665+0000", 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3640000000000002e-06 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450671+0000", 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450675+0000", 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466121+0000", 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015446051000000001 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466193+0000", 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1653e-05 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466270+0000", 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6756999999999997e-05 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29888 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0385710199999991, 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015745616, 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29888 2024-09-13T09:26:33.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450690+0000", 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450693+0000", 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-06 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450699+0000", 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0009999999999998e-06 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450704+0000", 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466289+0000", 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015584242999999999 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466353+0000", 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4738000000000003e-05 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466436+0000", 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3151000000000004e-05 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29889 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0382254540000009, 2024-09-13T09:26:33.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015595533999999999, 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29889 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451035+0000", 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451040+0000", 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:26:33.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451048+0000", 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7749999999999993e-06 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451054+0000", 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.314e-06 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466455+0000", 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015401141 2024-09-13T09:26:33.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466543+0000", 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7739999999999997e-05 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466632+0000", 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8672999999999996e-05 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29890 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0382018479999999, 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015753145, 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29890 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451059+0000", 2024-09-13T09:26:33.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451063+0000", 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451068+0000", 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6609999999999997e-06 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451073+0000", 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5120000000000002e-06 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466655+0000", 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015581858000000001 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466728+0000", 2024-09-13T09:26:33.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2912999999999998e-05 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466813+0000", 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5563999999999996e-05 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29891 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0381848609999995, 2024-09-13T09:26:33.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015920059, 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29891 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451079+0000", 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451084+0000", 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3260000000000003e-06 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451089+0000", 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6380000000000003e-06 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466832+0000", 2024-09-13T09:26:33.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015743549999999999 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466910+0000", 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7483000000000003e-05 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466997+0000", 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7146999999999998e-05 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29921 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 90112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0373389720000006, 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015647616, 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29921 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451922+0000", 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451925+0000", 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.334e-06 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451930+0000", 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9760000000000003e-06 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451935+0000", 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467403+0000", 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015468328 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467470+0000", 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6792999999999996e-05 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467570+0000", 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100658 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29922 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0373210420000003, 2024-09-13T09:26:33.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01583321, 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29922 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451940+0000", 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451943+0000", 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7760000000000002e-06 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451948+0000", 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451953+0000", 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467596+0000", 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015643106 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467678+0000", 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2338999999999999e-05 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467774+0000", 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5891999999999996e-05 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29923 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0373051610000008, 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016026886000000001, 2024-09-13T09:26:33.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29923 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451956+0000", 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451969+0000", 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2823000000000001e-05 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451976+0000", 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451981+0000", 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4650000000000002e-06 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467800+0000", 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015818412 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467888+0000", 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8256000000000003e-05 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467984+0000", 2024-09-13T09:26:33.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5680999999999994e-05 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29924 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 147456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0372763270000007, 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016203393999999999, 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.568 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29924 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451988+0000", 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7920000000000001e-06 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451993+0000", 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2979999999999999e-06 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451998+0000", 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.656e-06 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468003+0000", 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016005275999999999 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468093+0000", 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9530999999999994e-05 2024-09-13T09:26:33.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468189+0000", 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6356999999999999e-05 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29925 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0372605830000001, 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016402086999999999, 2024-09-13T09:26:33.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29925 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452003+0000", 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:26:33.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452008+0000", 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452012+0000", 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1749999999999998e-06 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468216+0000", 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016204309 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468307+0000", 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0917999999999999e-05 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468403+0000", 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6043000000000002e-05 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29926 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 204800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0372458370000004, 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016580355000000001, 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29926 2024-09-13T09:26:33.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:33.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452018+0000", 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3099999999999999e-06 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452023+0000", 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4149999999999998e-06 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452028+0000", 2024-09-13T09:26:33.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3730000000000003e-06 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468428+0000", 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016400237000000002 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468512+0000", 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4182000000000001e-05 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468596+0000", 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169e-05 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29927 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0372300419999991, 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016758252000000001, 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29927 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:33.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452031+0000", 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452034+0000", 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452039+0000", 2024-09-13T09:26:33.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.378e-06 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452043+0000", 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468625+0000", 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581722 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468701+0000", 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5909999999999997e-05 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468790+0000", 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8980999999999994e-05 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:30010 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:33.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0347127809999996, 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015722454, 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 30010 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:33.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454551+0000", 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454556+0000", 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:33.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454559+0000", 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3220000000000001e-06 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470137+0000", 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015577860000000001 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470210+0000", 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2986999999999998e-05 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:33.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470271+0000", 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1247999999999994e-05 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:33.590 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:33.590 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:26:33.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:33.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:33.593 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:33.583+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:33.693 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:33.692+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:33.750 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:33.750 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:33.750 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:33.750 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:33.761 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:33.761 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:33.780 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T09:26:33.783 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:33.804 INFO:teuthology.orchestra.run.smithi063.stdout:2009 punch from 0x8bd490e to 0x8be3595, (0xec87 bytes) 2024-09-13T09:26:33.805 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:33.804+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:33.808 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:33.811 INFO:teuthology.orchestra.run.smithi063.stdout:2010 read 0x1cdc3a6 thru 0x1ce8dcd (0xca28 bytes) 2024-09-13T09:26:33.813 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:33.813 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:33.813 INFO:teuthology.orchestra.run.smithi063.stdout:2013 write 0x2bd2465 thru 0x2bd5a56 (0x35f2 bytes) 2024-09-13T09:26:33.826 INFO:teuthology.orchestra.run.smithi063.stdout:2014 punch from 0x1372517 to 0x137f472, (0xcf5b bytes) 2024-09-13T09:26:33.826 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:33.826 INFO:teuthology.orchestra.run.smithi063.stdout:2017 writesame 0x8d3eb0c thru 0x8d40c03 data_size 0x14(0x20f8 bytes) 2024-09-13T09:26:33.834 INFO:teuthology.orchestra.run.smithi063.stdout:2018 writesame 0x314a08e thru 0x314a9fd data_size 0x8(0x970 bytes) 2024-09-13T09:26:33.834 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:33.854 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:33.855 INFO:teuthology.orchestra.run.smithi063.stdout:2019 write 0xb08feb3 thru 0xb096af6 (0x6c44 bytes) 2024-09-13T09:26:33.865 INFO:teuthology.orchestra.run.smithi063.stdout:2020 clone 18 order 21 su 8192 sc 7 2024-09-13T09:26:33.869 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:33.869 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:33.876 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:33.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:33.894 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:33.887+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:33.899 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:33.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:33.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:33.987 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:33.988 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:33.995 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:33.991+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:33.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:34.096 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:34.091+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:34.105 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.105 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.115 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:34.197 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:34.192+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:34.221 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.221 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:34.298 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:34.295+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:34.337 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.337 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:34.399 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:34.396+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:34.465 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.465 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.475 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:26:34.499 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:34.491+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:34.581 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.581 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:34.600 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:34.596+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:34.621 INFO:teuthology.orchestra.run.smithi063.stdout:flattening image image_client.0-clone17 2024-09-13T09:26:34.701 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:34.695+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.708 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.719 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:34.803 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:34.800+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:34.827 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:34.883 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:34.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:34.929 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:34.923+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:35.002 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.002 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.005 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:35.000+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:35.013 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:35.106 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:35.099+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:35.121 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.121 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.133 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:35.207 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:35.204+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:35.237 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T09:26:35.293 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.293 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:35.308 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:35.299+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:35.409 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:35.408+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:35.419 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.420 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.430 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:35.511 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:35.503+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:35.524 INFO:teuthology.orchestra.run.smithi063.stdout:2022 read 0x1043176 thru 0x104fe33 (0xccbe bytes) 2024-09-13T09:26:35.536 INFO:teuthology.orchestra.run.smithi063.stdout:2024 write 0x5495d84 thru 0x549d4a1 (0x771e bytes) 2024-09-13T09:26:35.539 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.539 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:35.551 INFO:teuthology.orchestra.run.smithi063.stdout:2025 punch from 0x8c0f6b4 to 0x8c18fe8, (0x9934 bytes) 2024-09-13T09:26:35.551 INFO:teuthology.orchestra.run.smithi063.stdout:2026 read 0x5b08336 thru 0x5b17749 (0xf414 bytes) 2024-09-13T09:26:35.554 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:35.554 INFO:teuthology.orchestra.run.smithi063.stdout:2029 punch from 0x1223377 to 0x1229fd9, (0x6c62 bytes) 2024-09-13T09:26:35.554 INFO:teuthology.orchestra.run.smithi063.stdout:2030 punch from 0x183ea51 to 0x183f50e, (0xabd bytes) 2024-09-13T09:26:35.554 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:35.554 INFO:teuthology.orchestra.run.smithi063.stdout:2032 clone 19 order 23 su 524288 sc 11 2024-09-13T09:26:35.612 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:35.603+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:35.671 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.671 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:35.713 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:35.707+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:35.785 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.785 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:35.814 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:35.807+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:35.901 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:35.901 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:35.905 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:35.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:35.949 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:35.943+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:36.013 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:36.013 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:36.016 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:36.007+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:36.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:36.066 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:36.112 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:36.117 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:36.107+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:36.132 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:36.132 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:36.137 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:36.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:36.162 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:36.185 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:36.208 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:36.209 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:36.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:36.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:36.217 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:36.216+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:36.246 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:36.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.779512013999998, 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:36.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:36.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.755169752, 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:36.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.750801389000003, 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:36.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:36.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:36.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.387372730999999, 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:36.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:36.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.551062250000001, 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:36.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:36.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:36.283 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.298056222, 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:36.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29819 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.793819084000001, 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022753847000000001, 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29819 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450415+0000", 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450418+0000", 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450423+0000", 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450428+0000", 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463252+0000", 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012823467999999999 2024-09-13T09:26:36.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463451+0000", 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019896599999999999 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473170+0000", 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009718829 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29887 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.793571427, 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015606436, 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29887 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450663+0000", 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450665+0000", 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3640000000000002e-06 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450671+0000", 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450675+0000", 2024-09-13T09:26:36.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466121+0000", 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015446051000000001 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466193+0000", 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1653e-05 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466270+0000", 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6756999999999997e-05 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29888 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 462848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.301 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.79354408, 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015745616, 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29888 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450691+0000", 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450690+0000", 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450693+0000", 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0369999999999999e-06 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450699+0000", 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0009999999999998e-06 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450704+0000", 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0520000000000001e-06 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466289+0000", 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015584242999999999 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466353+0000", 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4738000000000003e-05 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466436+0000", 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3151000000000004e-05 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29889 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.793198514, 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015595533999999999, 2024-09-13T09:26:36.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29889 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451036+0000", 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451035+0000", 2024-09-13T09:26:36.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451040+0000", 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451048+0000", 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7749999999999993e-06 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451054+0000", 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.314e-06 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466455+0000", 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015401141 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466543+0000", 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7739999999999997e-05 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466632+0000", 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8672999999999996e-05 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29890 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.793174907999999, 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015753145, 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29890 2024-09-13T09:26:36.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451060+0000", 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451059+0000", 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451063+0000", 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451068+0000", 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6609999999999997e-06 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451073+0000", 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5120000000000002e-06 2024-09-13T09:26:36.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466655+0000", 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015581858000000001 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466728+0000", 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2912999999999998e-05 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466813+0000", 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5563999999999996e-05 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29891 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 520192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.793157921000001, 2024-09-13T09:26:36.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015920059, 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29891 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:36.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451077+0000", 2024-09-13T09:26:36.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451079+0000", 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451084+0000", 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3260000000000003e-06 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451089+0000", 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6380000000000003e-06 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466832+0000", 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015743549999999999 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466910+0000", 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7483000000000003e-05 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.466997+0000", 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7146999999999998e-05 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29921 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 90112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792312032, 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015647616, 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29921 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451923+0000", 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451922+0000", 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451925+0000", 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.334e-06 2024-09-13T09:26:36.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451930+0000", 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9760000000000003e-06 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451935+0000", 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8980000000000004e-06 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467403+0000", 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015468328 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467470+0000", 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6792999999999996e-05 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467570+0000", 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100658 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29922 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792294102, 2024-09-13T09:26:36.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01583321, 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29922 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:36.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451941+0000", 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451940+0000", 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451943+0000", 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7760000000000002e-06 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451948+0000", 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451953+0000", 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4329999999999997e-06 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467596+0000", 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015643106 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467678+0000", 2024-09-13T09:26:36.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2338999999999999e-05 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467774+0000", 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5891999999999996e-05 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29923 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 118784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792278221, 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016026886000000001, 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29923 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:36.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451957+0000", 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451956+0000", 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451969+0000", 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2823000000000001e-05 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451976+0000", 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451981+0000", 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4650000000000002e-06 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467800+0000", 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015818412 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467888+0000", 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8256000000000003e-05 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.467984+0000", 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5680999999999994e-05 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29924 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 147456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792249387, 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016203393999999999, 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29924 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:36.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451985+0000", 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451988+0000", 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7920000000000001e-06 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451993+0000", 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2979999999999999e-06 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.451998+0000", 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.656e-06 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468003+0000", 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016005275999999999 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468093+0000", 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9530999999999994e-05 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468189+0000", 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6356999999999999e-05 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29925 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 176128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792233642999999, 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016402086999999999, 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29925 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:36.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452001+0000", 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452003+0000", 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:26:36.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452008+0000", 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452012+0000", 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1749999999999998e-06 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468216+0000", 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016204309 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468307+0000", 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0917999999999999e-05 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468403+0000", 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6043000000000002e-05 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29926 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 204800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792218897, 2024-09-13T09:26:36.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016580355000000001, 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29926 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452016+0000", 2024-09-13T09:26:36.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452018+0000", 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3099999999999999e-06 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452023+0000", 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4149999999999998e-06 2024-09-13T09:26:36.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452028+0000", 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3730000000000003e-06 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468428+0000", 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016400237000000002 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468512+0000", 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4182000000000001e-05 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468596+0000", 2024-09-13T09:26:36.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169e-05 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29927 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.792203102, 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016758252000000001, 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29927 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452031+0000", 2024-09-13T09:26:36.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452034+0000", 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452039+0000", 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.378e-06 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452043+0000", 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468625+0000", 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581722 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468701+0000", 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5909999999999997e-05 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468790+0000", 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8980999999999994e-05 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.363 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:30010 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 778240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.789685841000001, 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015722454, 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 30010 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454549+0000", 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454551+0000", 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454556+0000", 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454559+0000", 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3220000000000001e-06 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470137+0000", 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015577860000000001 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470210+0000", 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2986999999999998e-05 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.470271+0000", 2024-09-13T09:26:36.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1247999999999994e-05 2024-09-13T09:26:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.369 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:36.371 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:36.371 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:36.368+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:36.471 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:36.468+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:36.512 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:26:36.532 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:36.532 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:36.533 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:36.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:36.571 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:36.568+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:36.620 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone18 from 0xb096af7 (overlap 0xb096af7) to 0x8d45592 2024-09-13T09:26:36.662 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:36.662 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:36.662 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:36.672 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:36.668+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:36.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:36.773 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:36.768+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:36.829 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:36.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.267316557999997, 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:36.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29295 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.385032582999999, 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031712789999999998, 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29295 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:36.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441416+0000", 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441422+0000", 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.592e-06 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441427+0000", 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.453205+0000", 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011778116999999999 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454881+0000", 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016761879999999999 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473127+0000", 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018246294 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29497 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381986634, 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024159292999999998, 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29497 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444462+0000", 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5129999999999999e-06 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444468+0000", 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4009999999999996e-06 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444475+0000", 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8990000000000001e-06 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.465234+0000", 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020758316999999998 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.465299+0000", 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4873e-05 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468620+0000", 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033208769999999999 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29498 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381967418, 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024172662000000001, 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29498 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444481+0000", 2024-09-13T09:26:36.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444486+0000", 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1070000000000004e-06 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444491+0000", 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2379999999999999e-06 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468645+0000", 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024153813999999999 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468652+0000", 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6290000000000001e-06 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29499 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381952412, 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024184213, 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29499 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444496+0000", 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444502+0000", 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444506+0000", 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.279e-06 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468672+0000", 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02416565 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468679+0000", 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9959999999999996e-06 2024-09-13T09:26:36.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29500 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381934081000001, 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024191877000000001, 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29500 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444515+0000", 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1909999999999999e-06 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444520+0000", 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6230000000000002e-06 2024-09-13T09:26:36.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444525+0000", 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5040000000000004e-06 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468698+0000", 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024173310999999999 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468705+0000", 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.545e-06 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29501 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381918303999999, 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024199919, 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29501 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444528+0000", 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444531+0000", 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.311e-06 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444536+0000", 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1880000000000001e-06 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444540+0000", 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4009999999999999e-06 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468722+0000", 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024182209 2024-09-13T09:26:36.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468729+0000", 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29502 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381902885000001, 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024207818999999998, 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29502 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:36.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444549+0000", 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4929999999999998e-06 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444554+0000", 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444559+0000", 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0980000000000004e-06 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468745+0000", 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024186736 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468752+0000", 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5220000000000004e-06 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29503 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.38188441, 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024211703000000001, 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29503 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444564+0000", 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000002e-06 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444570+0000", 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4550000000000003e-06 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444574+0000", 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4129999999999999e-06 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468768+0000", 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024193840000000001 2024-09-13T09:26:36.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468774+0000", 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29504 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381868898, 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024219068999999999, 2024-09-13T09:26:36.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29504 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444580+0000", 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444585+0000", 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4260000000000002e-06 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444590+0000", 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2289999999999997e-06 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468791+0000", 2024-09-13T09:26:36.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024201139999999999 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468797+0000", 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29542 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381853491999999, 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024226968000000002, 2024-09-13T09:26:36.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29542 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444601+0000", 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444607+0000", 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9660000000000001e-06 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444611+0000", 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6920000000000001e-06 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468814+0000", 2024-09-13T09:26:36.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024202893 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468820+0000", 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29544 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381521029, 2024-09-13T09:26:36.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023918539999999999, 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29544 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444925+0000", 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444933+0000", 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3640000000000001e-06 2024-09-13T09:26:36.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444941+0000", 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9449999999999993e-06 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444947+0000", 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468838+0000", 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023891194000000001 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468844+0000", 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3380000000000001e-06 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29545 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:36.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381495408999999, 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023916804, 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29545 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444954+0000", 2024-09-13T09:26:36.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.615e-06 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444959+0000", 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444964+0000", 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468862+0000", 2024-09-13T09:26:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023898189 2024-09-13T09:26:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468868+0000", 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4330000000000001e-06 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29546 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381479484, 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023923954000000001, 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29546 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:36.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444969+0000", 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444975+0000", 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4940000000000003e-06 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444979+0000", 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5379999999999996e-06 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468885+0000", 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023905531000000001 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468891+0000", 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.455e-06 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29547 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381463602, 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023930462, 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29547 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444985+0000", 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3290000000000001e-06 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444992+0000", 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709999999999996e-06 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444997+0000", 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3409999999999996e-06 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468907+0000", 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023910579000000001 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468914+0000", 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4019999999999999e-06 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29548 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381446209, 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023936915, 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29548 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445000+0000", 2024-09-13T09:26:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445003+0000", 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445008+0000", 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445017+0000", 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.533e-06 2024-09-13T09:26:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468931+0000", 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023913750000000001 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468938+0000", 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29549 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381424943000001, 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023939946, 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29549 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445024+0000", 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4930000000000001e-06 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445030+0000", 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.088e-06 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445035+0000", 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468955+0000", 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023920675999999998 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468962+0000", 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29550 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381408233, 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023946261999999999, 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29550 2024-09-13T09:26:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445038+0000", 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445041+0000", 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3329999999999999e-06 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445046+0000", 2024-09-13T09:26:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445051+0000", 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5149999999999997e-06 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468979+0000", 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023927166 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468985+0000", 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29551 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.381376775, 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023937653, 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29551 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445073+0000", 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0070000000000002e-06 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445079+0000", 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445084+0000", 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8169999999999998e-06 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.469002+0000", 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023917559000000001 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.469008+0000", 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1319999999999999e-06 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:35809 3.3 e52/25 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head v 52'133, mlcod=52'130)", 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9879435990000003, 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03073795, 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838501+0000", 2024-09-13T09:26:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853352+0000", 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014850436999999999 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853379+0000", 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7291e-05 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853393+0000", 2024-09-13T09:26:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853498+0000", 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010524899999999999 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853523+0000", 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4800999999999999e-05 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869155+0000", 2024-09-13T09:26:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015632732999999999 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869241+0000", 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5935999999999998e-05 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:36.916 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:36.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:36.918 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:36.911+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:36.935 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T09:26:37.019 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:37.016+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:37.033 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:39781 3.12 3:4cea1d99:::rbd_data.111818d3d5a9.0000000000000000:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.029849+0000", 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0029481590000000001, 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029671300000000001, 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 39781 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029849+0000", 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029849+0000", 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029846+0000", 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029861+0000", 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4799e-05 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029892+0000", 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1006000000000001e-05 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029908+0000", 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.615e-05 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030038+0000", 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000130197 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030744+0000", 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00070510500000000003 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:39782 3.d 3:b0cd2e84:::rbd_data.111818d3d5a9.0000000000000001:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.029931+0000", 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0028660140000000001, 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002927378, 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 39782 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029931+0000", 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029931+0000", 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029930+0000", 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029935+0000", 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0170000000000004e-06 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029947+0000", 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1652e-05 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.029961+0000", 2024-09-13T09:26:37.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4100000000000001e-05 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030140+0000", 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000179192 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030389+0000", 2024-09-13T09:26:37.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024828900000000001 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:39783 3.13 3:ce4788df:::rbd_data.111818d3d5a9.0000000000000002:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.030015+0000", 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0027816859999999998, 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002876913, 2024-09-13T09:26:37.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 39783 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030015+0000", 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030015+0000", 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030014+0000", 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030022+0000", 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5100000000000001e-06 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030075+0000", 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2709e-05 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030089+0000", 2024-09-13T09:26:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4762e-05 2024-09-13T09:26:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030232+0000", 2024-09-13T09:26:37.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000142927 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030441+0000", 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000208519 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:39786 3.d 3.14f731cd (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.030618+0000", 2024-09-13T09:26:37.048 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00217929, 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023055630000000001, 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 39786 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030618+0000", 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030618+0000", 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030617+0000", 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030655+0000", 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8742999999999998e-05 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030676+0000", 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0789e-05 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030692+0000", 2024-09-13T09:26:37.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.609e-05 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:39792 3.15 3.7f23f6d5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.030884+0000", 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0019132349999999999, 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020665039999999998, 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:37.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 39792 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030884+0000", 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030884+0000", 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030883+0000", 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030889+0000", 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6089999999999997e-06 2024-09-13T09:26:37.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.030904+0000", 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5027e-05 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031368+0000", 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00046339999999999999 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:39794 3.d 3.1bfc30d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.031385+0000", 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0014117369999999999, 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015902469999999999, 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 39794 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031385+0000", 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031385+0000", 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031384+0000", 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031391+0000", 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0890000000000002e-06 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031406+0000", 2024-09-13T09:26:37.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.507e-05 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.031416+0000", 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6469999999999993e-06 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 6 2024-09-13T09:26:37.059 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:37.061 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:37.119 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:37.116+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:37.170 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:37.171 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:37.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:37.221 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:37.220+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:37.287 INFO:teuthology.orchestra.run.smithi063.stdout:2033 writesame 0xb5fdd7a thru 0xb60b178 data_size 0x1(0xd3ff bytes) 2024-09-13T09:26:37.302 INFO:teuthology.orchestra.run.smithi063.stdout:2034 write 0x897d685 thru 0x897f51b (0x1e97 bytes) 2024-09-13T09:26:37.321 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:37.320+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:37.330 INFO:teuthology.orchestra.run.smithi063.stdout:2035 write 0x78055fb thru 0x780dfe3 (0x89e9 bytes) 2024-09-13T09:26:37.347 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:37.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 54.383983991000001, 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:37.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:37.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:37.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4166.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.387594536000002, 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044918628000000002, 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4166", 2024-09-13T09:26:37.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/1560218502", 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957404+0000", 2024-09-13T09:26:37.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957403+0000", 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957410+0000", 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957421+0000", 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0882e-05 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957429+0000", 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957564+0000", 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013534300000000001 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.957590+0000", 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5775999999999999e-05 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998028+0000", 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.040437256999999997 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998374+0000", 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034634200000000003 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.998620+0000", 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024635100000000001 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.001530+0000", 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029097649999999999 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002075+0000", 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054477099999999999 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002224+0000", 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014908400000000001 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:44.002323+0000", 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8954000000000002e-05 2024-09-13T09:26:37.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:4479 3.d 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.509238571000001, 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051456631000000003, 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4479 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835760+0000", 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.835751+0000", 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999909 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854271+0000", 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01851972 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854305+0000", 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4258000000000003e-05 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854317+0000", 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2013e-05 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854487+0000", 2024-09-13T09:26:37.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017018 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.854768+0000", 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028067700000000001 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.855666+0000", 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000898404 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:37.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.881583+0000", 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025916947999999999 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887097+0000", 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055134479999999998 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887161+0000", 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4077000000000003e-05 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.887217+0000", 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6128000000000002e-05 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29813 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 417792~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475915+0000", 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.869084058, 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041775513, 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29813 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475915+0000", 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475915+0000", 2024-09-13T09:26:37.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475914+0000", 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475917+0000", 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.79e-06 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475923+0000", 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9020000000000004e-06 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475928+0000", 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-06 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517554+0000", 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041626412000000002 2024-09-13T09:26:37.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517612+0000", 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7411999999999999e-05 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517690+0000", 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8405999999999994e-05 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29814 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 446464~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.869066598, 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041923992, 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29814 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475932+0000", 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475934+0000", 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4159999999999998e-06 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475939+0000", 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-13T09:26:37.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475943+0000", 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517722+0000", 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041778932999999997 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517777+0000", 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4843999999999999e-05 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517856+0000", 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8813999999999995e-05 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.375 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29815 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 475136~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.869051265, 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042062642999999997, 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29815 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475947+0000", 2024-09-13T09:26:37.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475949+0000", 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1830000000000001e-06 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475954+0000", 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1220000000000003e-06 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475960+0000", 2024-09-13T09:26:37.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9660000000000001e-06 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517875+0000", 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041914712999999999 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.517932+0000", 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7411000000000003e-05 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518010+0000", 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7585999999999999e-05 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29816 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 503808~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.869034394, 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042212265999999998, 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29816 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475964+0000", 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475966+0000", 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.165e-06 2024-09-13T09:26:37.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475971+0000", 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0409999999999997e-06 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475976+0000", 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518042+0000", 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042066192000000002 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518098+0000", 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6347000000000001e-05 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518177+0000", 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8227999999999997e-05 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29871 3.f 3:f1039636:::rbd_data.111811db2be7.0000000000000001:f [read 20480~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.475980+0000", 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.869019121000001, 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04234835, 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29871 2024-09-13T09:26:37.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475980+0000", 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475980+0000", 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475979+0000", 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.475999+0000", 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9633e-05 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476005+0000", 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2740000000000004e-06 2024-09-13T09:26:37.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476010+0000", 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518195+0000", 2024-09-13T09:26:37.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042184455000000003 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518252+0000", 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7627000000000002e-05 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518328+0000", 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5717999999999996e-05 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29873 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868984519, 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042371669000000001, 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29873 2024-09-13T09:26:37.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476014+0000", 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476016+0000", 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3769999999999999e-06 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476021+0000", 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476027+0000", 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2049999999999999e-06 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518371+0000", 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042343486 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518386+0000", 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5121e-05 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29874 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868967286, 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042386539000000001, 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29874 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476031+0000", 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476033+0000", 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.289e-06 2024-09-13T09:26:37.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476039+0000", 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476043+0000", 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518408+0000", 2024-09-13T09:26:37.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042364786000000001 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518418+0000", 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7010000000000008e-06 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29875 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868946604, 2024-09-13T09:26:37.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042388589999999997, 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29875 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476052+0000", 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476054+0000", 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476060+0000", 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.356e-06 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476066+0000", 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4489999999999996e-06 2024-09-13T09:26:37.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518433+0000", 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042366899999999999 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518441+0000", 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7980000000000006e-06 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29880 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868118365000001, 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041772767000000002, 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29880 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.403 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476880+0000", 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476883+0000", 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5639999999999999e-06 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476889+0000", 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1569999999999997e-06 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476908+0000", 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9015999999999999e-05 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518646+0000", 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041737784 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518653+0000", 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5569999999999998e-06 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.406 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29892 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476926+0000", 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868072784000001, 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041808019000000002, 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29892 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476926+0000", 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476926+0000", 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476925+0000", 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476928+0000", 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1039999999999998e-06 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476936+0000", 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1500000000000002e-06 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476957+0000", 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1090999999999998e-05 2024-09-13T09:26:37.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518701+0000", 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041744548999999999 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518734+0000", 2024-09-13T09:26:37.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2641999999999997e-05 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29893 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868036342, 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041786809000000001, 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29893 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476962+0000", 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476971+0000", 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1749999999999994e-06 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476979+0000", 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1499999999999999e-06 2024-09-13T09:26:37.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476984+0000", 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2830000000000001e-06 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518745+0000", 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041760799000000001 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518749+0000", 2024-09-13T09:26:37.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9369999999999997e-06 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29894 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.868009511, 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041820164999999999, 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29894 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476989+0000", 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.476991+0000", 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6029999999999999e-06 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477000+0000", 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1219999999999995e-06 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477006+0000", 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0379999999999996e-06 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518795+0000", 2024-09-13T09:26:37.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041789735000000001 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518809+0000", 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4038e-05 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29895 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477010+0000", 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.867989079999999, 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041816114000000001, 2024-09-13T09:26:37.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29895 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477010+0000", 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477010+0000", 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477009+0000", 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477012+0000", 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.26e-06 2024-09-13T09:26:37.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477018+0000", 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477023+0000", 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518822+0000", 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041798163999999999 2024-09-13T09:26:37.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518826+0000", 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1049999999999997e-06 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29896 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.867971502, 2024-09-13T09:26:37.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041844904000000002, 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29896 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477027+0000", 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477029+0000", 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477034+0000", 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9949999999999996e-06 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477038+0000", 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2969999999999997e-06 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518849+0000", 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041810711 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518872+0000", 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2909e-05 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29897 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.867956232000001, 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856206, 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29897 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.428 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477042+0000", 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477050+0000", 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0679999999999997e-06 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477056+0000", 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.57e-06 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477060+0000", 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1810000000000003e-06 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518891+0000", 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041830621999999998 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518899+0000", 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0919999999999998e-06 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29898 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477065+0000", 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.867933990999999, 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041867695000000003, 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29898 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477065+0000", 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477065+0000", 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477064+0000", 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477072+0000", 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2579999999999998e-06 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477077+0000", 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.536e-06 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477083+0000", 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7710000000000003e-06 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518924+0000", 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041840914 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518932+0000", 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5429999999999994e-06 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29899 3.f 3.6c69c08f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.477103+0000", 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.867895679, 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041865687999999998, 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29899 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477103+0000", 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477103+0000", 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:37.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477102+0000", 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477105+0000", 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.05e-06 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477111+0000", 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8200000000000002e-06 2024-09-13T09:26:37.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.477116+0000", 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5029999999999999e-06 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518955+0000", 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041839297999999997 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:37.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.518969+0000", 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3682999999999999e-05 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:37.440 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:37.441 INFO:teuthology.orchestra.run.smithi063.stdout:2036 trunc from 0xb60b179 to 0x908effe 2024-09-13T09:26:37.442 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:37.442 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:37.435+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:37.530 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:26:37.542 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:37.535+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:37.558 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:37.558 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:37.559 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:37.559 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:37.570 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:37.574 INFO:teuthology.orchestra.run.smithi063.stdout:2041 write 0xdf45563 thru 0xdf4e289 (0x8d27 bytes) 2024-09-13T09:26:37.579 INFO:teuthology.orchestra.run.smithi063.stdout:2043 read 0xbe00f0 thru 0xbe65d1 (0x64e2 bytes) 2024-09-13T09:26:37.581 INFO:teuthology.orchestra.run.smithi063.stdout:2046 punch from 0x2d8521c to 0x2d86d63, (0x1b47 bytes) 2024-09-13T09:26:37.589 INFO:teuthology.orchestra.run.smithi063.stdout:2047 read 0x7be7f6 thru 0x7ccb37 (0xe342 bytes) 2024-09-13T09:26:37.612 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.612 INFO:teuthology.orchestra.run.smithi063.stdout:2049 punch from 0x63e0e14 to 0x63f03af, (0xf59b bytes) 2024-09-13T09:26:37.617 INFO:teuthology.orchestra.run.smithi063.stdout:2050 read 0xbc9c611 thru 0xbca8fb0 (0xc9a0 bytes) 2024-09-13T09:26:37.617 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.617 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.617 INFO:teuthology.orchestra.run.smithi063.stdout:2056 read 0xd42623b thru 0xd431dfb (0xbbc1 bytes) 2024-09-13T09:26:37.617 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.618 INFO:teuthology.orchestra.run.smithi063.stdout:2058 read 0x7ad48e4 thru 0x7ae0a20 (0xc13d bytes) 2024-09-13T09:26:37.626 INFO:teuthology.orchestra.run.smithi063.stdout:2059 punch from 0x6429df0 to 0x6438ade, (0xecee bytes) 2024-09-13T09:26:37.626 INFO:teuthology.orchestra.run.smithi063.stdout:2060 read 0x912d365 thru 0x9131a8c (0x4728 bytes) 2024-09-13T09:26:37.643 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:37.635+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:37.649 INFO:teuthology.orchestra.run.smithi063.stdout:2061 trunc from 0xdf4e28a to 0xc094d1b 2024-09-13T09:26:37.675 INFO:teuthology.orchestra.run.smithi063.stdout:2063 writesame 0x6ff6749 thru 0x6ff7a8e data_size 0x1(0x1346 bytes) 2024-09-13T09:26:37.680 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.680 INFO:teuthology.orchestra.run.smithi063.stdout:2065 read 0x261c640 thru 0x2627220 (0xabe1 bytes) 2024-09-13T09:26:37.682 INFO:teuthology.orchestra.run.smithi063.stdout:2066 writesame 0x8243c59 thru 0x8252b18 data_size 0x50(0xeec0 bytes) 2024-09-13T09:26:37.691 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:37.692 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:37.692 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:37.692 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:37.692 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:37.699 INFO:teuthology.orchestra.run.smithi063.stdout:2068 read 0x8e43936 thru 0x8e50832 (0xcefd bytes) 2024-09-13T09:26:37.700 INFO:teuthology.orchestra.run.smithi063.stdout:2069 read 0x2ab04de thru 0x2ab9243 (0x8d66 bytes) 2024-09-13T09:26:37.703 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:37.707 INFO:teuthology.orchestra.run.smithi063.stdout:2071 write 0x3f87a53 thru 0x3f8bca8 (0x4256 bytes) 2024-09-13T09:26:37.744 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:37.740+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:37.752 INFO:teuthology.orchestra.run.smithi063.stdout:2072 read 0x2e90631 thru 0x2e93199 (0x2b69 bytes) 2024-09-13T09:26:37.753 INFO:teuthology.orchestra.run.smithi063.stdout:2074 write 0x4d34d44 thru 0x4d40063 (0xb320 bytes) 2024-09-13T09:26:37.791 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-13T09:26:37.792 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:37.792 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-13T09:26:37.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-13T09:26:37.821 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:37.821 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:37.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:37.821 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:37.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:5 1.2 e18/16 1:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 53.868980477000001, 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00165924, 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949732+0000", 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949731+0000", 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949758+0000", 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7019999999999999e-05 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949785+0000", 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7419e-05 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949805+0000", 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0296e-05 2024-09-13T09:26:37.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.949949+0000", 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014343400000000001 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.950000+0000", 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1267999999999998e-05 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:37.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951329+0000", 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013283329999999999 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:43.951391+0000", 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2232999999999999e-05 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4211.0:7 1.2 e19/16 1:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.852422263999998, 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013847040000000001, 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966290+0000", 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966289+0000", 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966307+0000", 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7672e-05 2024-09-13T09:26:37.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966327+0000", 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9890000000000001e-05 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966346+0000", 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9338999999999999e-05 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966410+0000", 2024-09-13T09:26:37.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4325999999999994e-05 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.966431+0000", 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1191000000000001e-05 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967624+0000", 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011922020000000001 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:44.967674+0000", 2024-09-13T09:26:37.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0733999999999999e-05 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:37.834 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.617675005999999, 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002402941, 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.835 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 11 2024-09-13T09:26:37.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.836 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:37.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201037+0000", 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201034+0000", 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201051+0000", 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6935e-05 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201075+0000", 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3632e-05 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201101+0000", 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6415e-05 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201191+0000", 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9790000000000006e-05 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201416+0000", 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022460500000000001 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.201684+0000", 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026827200000000001 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.202899+0000", 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012148 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203339+0000", 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00044027200000000002 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203398+0000", 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8465999999999999e-05 2024-09-13T09:26:37.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203440+0000", 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2281999999999999e-05 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:11 3.1f e24/21 ondisk, result = 0)", 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.615465868999998, 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020497399999999999, 2024-09-13T09:26:37.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203246+0000", 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203245+0000", 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203252+0000", 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5749999999999997e-06 2024-09-13T09:26:37.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203266+0000", 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3909e-05 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203276+0000", 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0479000000000001e-05 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203324+0000", 2024-09-13T09:26:37.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7994000000000002e-05 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203334+0000", 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0389e-05 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.203451+0000", 2024-09-13T09:26:37.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011665799999999999 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:12 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'1, mlcod=0'0)", 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.611121343999997, 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001120151, 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207591+0000", 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207590+0000", 2024-09-13T09:26:37.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207601+0000", 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0886999999999999e-05 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207614+0000", 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.349e-05 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207623+0000", 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1179999999999998e-06 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207712+0000", 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8327999999999999e-05 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.207752+0000", 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0000999999999999e-05 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208676+0000", 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092465699999999995 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.208711+0000", 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4502000000000003e-05 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.594570365000003, 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00038220099999999998, 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 14 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.852 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224142+0000", 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224140+0000", 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224149+0000", 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8370000000000003e-06 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224169+0000", 2024-09-13T09:26:37.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0540999999999999e-05 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224189+0000", 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9446999999999999e-05 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224274+0000", 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5455000000000001e-05 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224399+0000", 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012455800000000001 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224524+0000", 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012512999999999999 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:37.856 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.593982357000002, 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059809700000000002, 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 15 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:37.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224730+0000", 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224728+0000", 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224737+0000", 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:26:37.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224755+0000", 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7734e-05 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224763+0000", 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1850000000000004e-06 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224844+0000", 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0541e-05 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.224965+0000", 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120856 2024-09-13T09:26:37.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.225328+0000", 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036304000000000002 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:25 3.18 e24/21 3:1a426f9e:::rbd_header.11182e510213:head v 24'2, mlcod=0'0)", 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.586276200999997, 2024-09-13T09:26:37.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010867100000000001, 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232436+0000", 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232434+0000", 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232447+0000", 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3149000000000001e-05 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232469+0000", 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1725999999999998e-05 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232478+0000", 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1980000000000007e-06 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232563+0000", 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4184999999999994e-05 2024-09-13T09:26:37.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.232611+0000", 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8195000000000002e-05 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233469+0000", 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085830499999999996 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:37.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:55.233522+0000", 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3402999999999997e-05 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:37.866 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:37.868 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:37.868 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:37.859+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:37.868 INFO:teuthology.orchestra.run.smithi063.stdout:2075 trunc from 0xc094d1b to 0xabe69af 2024-09-13T09:26:37.889 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.902 INFO:teuthology.orchestra.run.smithi063.stdout:2077 writesame 0xd2e195f thru 0xd2e858e data_size 0xc(0x6c30 bytes) 2024-09-13T09:26:37.902 INFO:teuthology.orchestra.run.smithi063.stdout:2078 writesame 0x448a47f thru 0x448c093 data_size 0xd(0x1c15 bytes) 2024-09-13T09:26:37.904 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:37.924 INFO:teuthology.orchestra.run.smithi063.stdout:2082 trunc from 0xd2e858f to 0x1b3af78 2024-09-13T09:26:37.968 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:37.959+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:37.994 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:37.994 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:37.994 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:38.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:38.069 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:38.063+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:38.108 INFO:teuthology.orchestra.run.smithi063.stdout:2083 trunc from 0x1b3af78 to 0xeb596c8 2024-09-13T09:26:38.120 INFO:teuthology.orchestra.run.smithi063.stdout:2084 read 0x87c4dc1 thru 0x87cff4c (0xb18c bytes) 2024-09-13T09:26:38.120 INFO:teuthology.orchestra.run.smithi063.stdout:2085 writesame 0x615495 thru 0x622811 data_size 0x1(0xd37d bytes) 2024-09-13T09:26:38.148 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:26:38.139+0000 7f00df3a9700 -1 mon.a@0(leader).osd e56 definitely_dead 0 2024-09-13T09:26:38.160 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.160 INFO:teuthology.orchestra.run.smithi063.stdout:2088 read 0x4abe969 thru 0x4ac07de (0x1e76 bytes) 2024-09-13T09:26:38.161 INFO:teuthology.orchestra.run.smithi063.stdout:2089 punch from 0x1e7923c to 0x1e7a736, (0x14fa bytes) 2024-09-13T09:26:38.162 INFO:teuthology.orchestra.run.smithi063.stdout:2091 read 0x805d15d thru 0x806b631 (0xe4d5 bytes) 2024-09-13T09:26:38.162 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.162 INFO:teuthology.orchestra.run.smithi063.stdout:2093 writesame 0xad2b8d thru 0xad5051 data_size 0x1(0x24c5 bytes) 2024-09-13T09:26:38.165 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:38.166 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:38.166 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:38.166 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:38.166 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:38.170 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:38.168+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:38.174 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.174 INFO:teuthology.orchestra.run.smithi063.stdout:2097 read 0xd6d692e thru 0xd6e09f9 (0xa0cc bytes) 2024-09-13T09:26:38.174 INFO:teuthology.orchestra.run.smithi063.stdout:2098 writesame 0x258b81e thru 0x259b51b data_size 0x5e(0xfcfe bytes) 2024-09-13T09:26:38.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:38.178 INFO:teuthology.orchestra.run.smithi063.stdout:2100 write 0x9673f93 thru 0x96817a2 (0xd810 bytes) 2024-09-13T09:26:38.186 INFO:teuthology.orchestra.run.smithi063.stdout:2101 write 0xfb1d35 thru 0xfba486 (0x8752 bytes) 2024-09-13T09:26:38.189 INFO:teuthology.orchestra.run.smithi063.stdout:2102 punch from 0x3381863 to 0x338bd8d, (0xa52a bytes) 2024-09-13T09:26:38.189 INFO:teuthology.orchestra.run.smithi063.stdout:2103 read 0x9b13ba3 thru 0x9b1e52c (0xa98a bytes) 2024-09-13T09:26:38.189 INFO:teuthology.orchestra.run.smithi063.stdout:2104 writesame 0x33c976f thru 0x33d2234 data_size 0x5d(0x8ac6 bytes) 2024-09-13T09:26:38.190 INFO:teuthology.orchestra.run.smithi063.stdout:2105 write 0xd2fa533 thru 0xd2fb639 (0x1107 bytes) 2024-09-13T09:26:38.195 INFO:teuthology.orchestra.run.smithi063.stdout:2107 writesame 0x7da4ad2 thru 0x7db388e data_size 0x1(0xedbd bytes) 2024-09-13T09:26:38.214 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:38.225 INFO:teuthology.orchestra.run.smithi063.stdout:2108 writesame 0xead9f53 thru 0xeae4585 data_size 0x1(0xa633 bytes) 2024-09-13T09:26:38.248 INFO:teuthology.orchestra.run.smithi063.stdout:2109 write 0x813f133 thru 0x8142ade (0x39ac bytes) 2024-09-13T09:26:38.254 INFO:teuthology.orchestra.run.smithi063.stdout:2110 read 0x47a202d thru 0x47ad533 (0xb507 bytes) 2024-09-13T09:26:38.255 INFO:teuthology.orchestra.run.smithi063.stdout:2111 read 0x785cb6 thru 0x793531 (0xd87c bytes) 2024-09-13T09:26:38.256 INFO:teuthology.orchestra.run.smithi063.stdout:2112 read 0x8d42b49 thru 0x8d49a41 (0x6ef9 bytes) 2024-09-13T09:26:38.271 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:38.268+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:38.280 INFO:teuthology.orchestra.run.smithi063.stdout:2113 trunc from 0xeb596c8 to 0x6d58dd 2024-09-13T09:26:38.283 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:38.283 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:38.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:38.283 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.325571684, 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:26:38.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:26:38.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38799 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.645979+0000", 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6334217600000001, 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030380556, 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.292 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38799 2024-09-13T09:26:38.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.292 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645979+0000", 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645979+0000", 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645979+0000", 2024-09-13T09:26:38.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645981+0000", 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2589999999999999e-06 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645986+0000", 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2270000000000001e-06 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645991+0000", 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676335+0000", 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03034359 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676360+0000", 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4871999999999998e-05 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.295 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38800 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6334056029999999, 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030430858000000002, 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.296 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38800 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645997+0000", 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3089999999999998e-06 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646003+0000", 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646008+0000", 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4649999999999996e-06 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676408+0000", 2024-09-13T09:26:38.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030399981 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676426+0000", 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8649000000000001e-05 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38801 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.300 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646012+0000", 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633389212, 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030498862000000002, 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38801 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646012+0000", 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646012+0000", 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646011+0000", 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646019+0000", 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1799999999999999e-06 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646025+0000", 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646030+0000", 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676495+0000", 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030465587999999998 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676511+0000", 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5265e-05 2024-09-13T09:26:38.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38802 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6333670119999999, 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030529230000000001, 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38802 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646036+0000", 2024-09-13T09:26:38.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.435e-06 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646042+0000", 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5589999999999996e-06 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646058+0000", 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6871000000000002e-05 2024-09-13T09:26:38.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676552+0000", 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030493466 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676563+0000", 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1303e-05 2024-09-13T09:26:38.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38803 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633327626, 2024-09-13T09:26:38.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030546878, 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38803 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.311 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646076+0000", 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.557e-06 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646084+0000", 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646088+0000", 2024-09-13T09:26:38.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676606+0000", 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030518749000000001 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676620+0000", 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3902e-05 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38804 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6333105670000001, 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030555835999999999, 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.317 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38804 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646092+0000", 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7659999999999999e-06 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646095+0000", 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:26:38.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646106+0000", 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1275e-05 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676640+0000", 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030533716999999998 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676646+0000", 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3609999999999997e-06 2024-09-13T09:26:38.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38805 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633287653, 2024-09-13T09:26:38.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030569447999999999, 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38805 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646120+0000", 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9619999999999996e-06 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646124+0000", 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.335e-06 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646129+0000", 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3830000000000002e-06 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676675+0000", 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030546923 2024-09-13T09:26:38.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676683+0000", 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2729999999999997e-06 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38806 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633269936, 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030602078000000001, 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38806 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:38.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646132+0000", 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.522e-06 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646135+0000", 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0079999999999998e-06 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646138+0000", 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676721+0000", 2024-09-13T09:26:38.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030583128000000001 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676733+0000", 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1999e-05 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38807 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6332607609999998, 2024-09-13T09:26:38.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622640999999999, 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38807 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646141+0000", 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.381e-06 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646144+0000", 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646147+0000", 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.525e-06 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676752+0000", 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030605591000000001 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676763+0000", 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0543999999999999e-05 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38912 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633252076, 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030664937, 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38912 2024-09-13T09:26:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646150+0000", 2024-09-13T09:26:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4899999999999999e-06 2024-09-13T09:26:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646154+0000", 2024-09-13T09:26:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8449999999999996e-06 2024-09-13T09:26:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646158+0000", 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.549e-06 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676800+0000", 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030641462000000001 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676814+0000", 2024-09-13T09:26:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3844e-05 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38913 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646163+0000", 2024-09-13T09:26:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633238134, 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030679339, 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38913 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646163+0000", 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646163+0000", 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646162+0000", 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646165+0000", 2024-09-13T09:26:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4229999999999999e-06 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646171+0000", 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646176+0000", 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676830+0000", 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030653762000000001 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676842+0000", 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2211e-05 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38914 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646190+0000", 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6332110520000001, 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030690914, 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38914 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646190+0000", 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646190+0000", 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646189+0000", 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646192+0000", 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9809999999999999e-06 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646199+0000", 2024-09-13T09:26:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9450000000000004e-06 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646204+0000", 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8860000000000003e-06 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676873+0000", 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030668805 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676881+0000", 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7959999999999996e-06 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38915 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646209+0000", 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6331924, 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030705501999999999, 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38915 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646209+0000", 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646209+0000", 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646208+0000", 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646211+0000", 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4260000000000002e-06 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646216+0000", 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7880000000000003e-06 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646221+0000", 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3930000000000001e-06 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676906+0000", 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030685632000000001 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676914+0000", 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38916 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6331759269999999, 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030739682000000001, 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38916 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646227+0000", 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646233+0000", 2024-09-13T09:26:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.083e-06 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646238+0000", 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7049999999999998e-06 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676950+0000", 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030711883999999998 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676965+0000", 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4854e-05 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6331590440000001, 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:26:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38918 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6330936650000001, 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030720447000000001, 2024-09-13T09:26:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38918 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646310+0000", 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6490000000000001e-06 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646319+0000", 2024-09-13T09:26:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6149999999999997e-06 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646324+0000", 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677015+0000", 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030691408 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677028+0000", 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2527000000000001e-05 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38919 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6330729800000001, 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030725642000000001, 2024-09-13T09:26:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38919 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646330+0000", 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4530000000000001e-06 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646337+0000", 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2069999999999998e-06 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646342+0000", 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8559999999999998e-06 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677041+0000", 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030698578000000001 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677054+0000", 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2896e-05 2024-09-13T09:26:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38920 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.633054848, 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030735129999999999, 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38920 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646348+0000", 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646354+0000", 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646359+0000", 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677074+0000", 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030715498000000001 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677081+0000", 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.092e-06 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6330384819999999, 2024-09-13T09:26:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:26:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:38.388 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:38.390 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:38.391 INFO:teuthology.orchestra.run.smithi063.stdout:2114 write 0x8356dbd thru 0x8364708 (0xd94c bytes) 2024-09-13T09:26:38.391 INFO:teuthology.orchestra.run.smithi063.stdout:2119 write 0x2f9836e thru 0x2f9af03 (0x2b96 bytes) 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:2120 read 0x613ae88 thru 0x6144e92 (0xa00b bytes) 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:2121 write 0x71a8eb1 thru 0x71abb3d (0x2c8d bytes) 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:2125 punch from 0x5420252 to 0x542b2a1, (0xb04f bytes) 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:2128 write 0x441147b thru 0x441af7e (0x9b04 bytes) 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:2134 writesame 0xc15872 thru 0xc16849 data_size 0x1a(0xfd8 bytes) 2024-09-13T09:26:38.392 INFO:teuthology.orchestra.run.smithi063.stdout:2136 write 0x6a415c1 thru 0x6a41d5b (0x79b bytes) 2024-09-13T09:26:38.393 INFO:teuthology.orchestra.run.smithi063.stdout:2137 writesame 0x1eb1df8 thru 0x1eb5540 data_size 0x1(0x3749 bytes) 2024-09-13T09:26:38.393 INFO:teuthology.orchestra.run.smithi063.stdout:2138 write 0x7c03037 thru 0x7c09361 (0x632b bytes) 2024-09-13T09:26:38.393 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:38.393 INFO:teuthology.orchestra.run.smithi063.stdout:2140 read 0x6620db0 thru 0x662ea64 (0xdcb5 bytes) 2024-09-13T09:26:38.393 INFO:teuthology.orchestra.run.smithi063.stdout:2141 write 0x28152b0 thru 0x281d256 (0x7fa7 bytes) 2024-09-13T09:26:38.394 INFO:teuthology.orchestra.run.smithi063.stdout:2143 writesame 0x7fb69cf thru 0x7fbe608 data_size 0x1(0x7c3a bytes) 2024-09-13T09:26:38.394 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:38.394 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:38.383+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:38.395 INFO:teuthology.orchestra.run.smithi063.stdout:2144 write 0x227f43e thru 0x228204c (0x2c0f bytes) 2024-09-13T09:26:38.395 INFO:teuthology.orchestra.run.smithi063.stdout:2146 punch from 0x6a1ced7 to 0x6a1e12e, (0x1257 bytes) 2024-09-13T09:26:38.405 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:38.415 INFO:teuthology.orchestra.run.smithi063.stdout:2147 write 0xc432f60 thru 0xc43e0c6 (0xb167 bytes) 2024-09-13T09:26:38.422 INFO:teuthology.orchestra.run.smithi063.stdout:2148 write 0x4466113 thru 0x4475eff (0xfded bytes) 2024-09-13T09:26:38.425 INFO:teuthology.orchestra.run.smithi063.stdout:2149 write 0xdbc3c7 thru 0xdc1b84 (0x57be bytes) 2024-09-13T09:26:38.429 INFO:teuthology.orchestra.run.smithi063.stdout:2150 write 0x797083c thru 0x7973a5b (0x3220 bytes) 2024-09-13T09:26:38.430 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:38.435 INFO:teuthology.orchestra.run.smithi063.stdout:2151 writesame 0x20169db thru 0x20265df data_size 0x1(0xfc05 bytes) 2024-09-13T09:26:38.454 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:38.481 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.482 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:38.482 INFO:teuthology.orchestra.run.smithi063.stdout:2155 punch from 0x3a4b83b to 0x3a545a5, (0x8d6a bytes) 2024-09-13T09:26:38.482 INFO:teuthology.orchestra.run.smithi063.stdout:2156 punch from 0x1644990 to 0x1645aef, (0x115f bytes) 2024-09-13T09:26:38.482 INFO:teuthology.orchestra.run.smithi063.stdout:2157 write 0xb289f34 thru 0xb293e75 (0x9f42 bytes) 2024-09-13T09:26:38.485 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:38.486 INFO:teuthology.orchestra.run.smithi063.stdout:2158 write 0x7cf2e0f thru 0x7cf55a2 (0x2794 bytes) 2024-09-13T09:26:38.489 INFO:teuthology.orchestra.run.smithi063.stdout:2159 write 0x1868b99 thru 0x1874cd2 (0xc13a bytes) 2024-09-13T09:26:38.490 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:38.483+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:38.491 INFO:teuthology.orchestra.run.smithi063.stdout:2160 punch from 0x7cbbf3b to 0x7cca9ae, (0xea73 bytes) 2024-09-13T09:26:38.492 INFO:teuthology.orchestra.run.smithi063.stdout:2162 trunc from 0xc43e0c7 to 0xb54f89f 2024-09-13T09:26:38.506 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:38.506 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:38.507 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:38.513 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:38.514 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:38.519 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:38.520 INFO:teuthology.orchestra.run.smithi063.stdout:2163 writesame 0xe5af956 thru 0xe5b4a89 data_size 0x2(0x5134 bytes) 2024-09-13T09:26:38.520 INFO:teuthology.orchestra.run.smithi063.stdout:2165 read 0x928f663 thru 0x92929ba (0x3358 bytes) 2024-09-13T09:26:38.527 INFO:teuthology.orchestra.run.smithi063.stdout:2166 trunc from 0xe5b4a8a to 0xbf037ba 2024-09-13T09:26:38.532 INFO:teuthology.orchestra.run.smithi063.stderr:marked in osd.0. 2024-09-13T09:26:38.543 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-13T09:26:38.592 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:38.588+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:38.618 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:38.618 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:38.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:38.693 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:38.688+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:38.732 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:38.732 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:38.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:38.793 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:38.792+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:38.858 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:38.858 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:38.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:38.894 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:38.892+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:38.978 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:38.978 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:38.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:39.030 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:39.028+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:39.097 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:39.092+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:39.140 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.140 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:39.198 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:39.196+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.252 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:39.299 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:39.296+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:39.371 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.371 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:39.426 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:39.424+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:39.485 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.485 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.495 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:39.501 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:39.495+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:39.546 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:26:39.602 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:39.600+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:39.609 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.609 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:39.703 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:39.695+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:39.720 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.720 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:39.804 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:39.800+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:39.837 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:39.837 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:39.846 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:39.905 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:39.899+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:40.002 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:40.002 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:40.006 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:40.004+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:40.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:40.107 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:40.104+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:40.116 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:40.116 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:40.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:40.208 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:40.199+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:40.276 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:40.276 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:40.287 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:40.308 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:40.304+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:40.405 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:40.405 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:40.409 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:40.403+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:40.415 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:40.510 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:40.508+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:40.515 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:40.520 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:40.521 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:40.531 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:40.543 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:26:40.569 INFO:teuthology.orchestra.run.smithi063.stdout:2167 write 0x13993c6 thru 0x139c1c4 (0x2dff bytes) 2024-09-13T09:26:40.571 INFO:teuthology.orchestra.run.smithi063.stdout:2169 read 0x3a7330c thru 0x3a76b6f (0x3864 bytes) 2024-09-13T09:26:40.572 INFO:teuthology.orchestra.run.smithi063.stdout:2170 read 0x114963a thru 0x114f438 (0x5dff bytes) 2024-09-13T09:26:40.572 INFO:teuthology.orchestra.run.smithi063.stdout:2171 write 0x9775484 thru 0x977a0b6 (0x4c33 bytes) 2024-09-13T09:26:40.577 INFO:teuthology.orchestra.run.smithi063.stdout:2172 read 0x2600c50 thru 0x260aaae (0x9e5f bytes) 2024-09-13T09:26:40.578 INFO:teuthology.orchestra.run.smithi063.stdout:2173 punch from 0x8fd9cac to 0x8fe53f5, (0xb749 bytes) 2024-09-13T09:26:40.578 INFO:teuthology.orchestra.run.smithi063.stdout:2176 write 0x9e7db76 thru 0x9e8089f (0x2d2a bytes) 2024-09-13T09:26:40.584 INFO:teuthology.orchestra.run.smithi063.stdout:2177 punch from 0x993c374 to 0x993dcda, (0x1966 bytes) 2024-09-13T09:26:40.596 INFO:teuthology.orchestra.run.smithi063.stdout:2178 write 0xc516ab0 thru 0xc52278e (0xbcdf bytes) 2024-09-13T09:26:40.603 INFO:teuthology.orchestra.run.smithi063.stdout:2179 read 0x3078732 thru 0x3081a8a (0x9359 bytes) 2024-09-13T09:26:40.604 INFO:teuthology.orchestra.run.smithi063.stdout:2180 read 0x7f5faef thru 0x7f6b272 (0xb784 bytes) 2024-09-13T09:26:40.604 INFO:teuthology.orchestra.run.smithi063.stdout:2181 read 0xed6863 thru 0xed99fc (0x319a bytes) 2024-09-13T09:26:40.605 INFO:teuthology.orchestra.run.smithi063.stdout:2182 punch from 0x13161a5 to 0x13230e8, (0xcf43 bytes) 2024-09-13T09:26:40.605 INFO:teuthology.orchestra.run.smithi063.stdout:2183 write 0x6e6105a thru 0x6e6f2b1 (0xe258 bytes) 2024-09-13T09:26:40.608 INFO:teuthology.orchestra.run.smithi063.stdout:2184 read 0x988a0ad thru 0x9892508 (0x845c bytes) 2024-09-13T09:26:40.609 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:40.611 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:40.608+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:40.616 INFO:teuthology.orchestra.run.smithi063.stdout:2187 trunc from 0xc52278f to 0x85e0316 2024-09-13T09:26:40.650 INFO:teuthology.orchestra.run.smithi063.stdout:2188 write 0xa0675aa thru 0xa07664e (0xf0a5 bytes) 2024-09-13T09:26:40.652 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:40.652 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:40.654 INFO:teuthology.orchestra.run.smithi063.stdout:2191 punch from 0x4018929 to 0x401dbf3, (0x52ca bytes) 2024-09-13T09:26:40.665 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:40.668 INFO:teuthology.orchestra.run.smithi063.stdout:2192 write 0xbd75e92 thru 0xbd816c6 (0xb835 bytes) 2024-09-13T09:26:40.675 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:40.676 INFO:teuthology.orchestra.run.smithi063.stdout:2194 read 0x13fa8bd thru 0x13fe2fa (0x3a3e bytes) 2024-09-13T09:26:40.677 INFO:teuthology.orchestra.run.smithi063.stdout:2195 read 0x5c58ef4 thru 0x5c68c21 (0xfd2e bytes) 2024-09-13T09:26:40.678 INFO:teuthology.orchestra.run.smithi063.stdout:2196 punch from 0x4fdc5fa to 0x4fe1a7e, (0x5484 bytes) 2024-09-13T09:26:40.678 INFO:teuthology.orchestra.run.smithi063.stdout:2197 writesame 0x20cbd0f thru 0x20d6656 data_size 0x4(0xa948 bytes) 2024-09-13T09:26:40.688 INFO:teuthology.orchestra.run.smithi063.stdout:2199 read 0x46e2cf6 thru 0x46f0a26 (0xdd31 bytes) 2024-09-13T09:26:40.688 INFO:teuthology.orchestra.run.smithi063.stdout:2201 clone 20 order 18 su 1024 sc 15 2024-09-13T09:26:40.699 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:40.712 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:40.708+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:40.718 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:40.741 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:40.763 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:40.775 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:40.775 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:40.775 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:40.775 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:40.785 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:40.788 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:40.812 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:40.812 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:40.813 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:40.808+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:40.817 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:40.817 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:40.897 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:40.898 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:40.898 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:40.898 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:40.898 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:40.909 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:40.914 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:40.907+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:41.015 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:41.012+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:41.030 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:41.030 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:41.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:41.030 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:41.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.071824147000001, 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:41.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:26:41.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:41.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:26:41.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38800 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3796580660000002, 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030430858000000002, 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38800 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645995+0000", 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.645997+0000", 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3089999999999998e-06 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646003+0000", 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:26:41.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646008+0000", 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4649999999999996e-06 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676408+0000", 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030399981 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676426+0000", 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8649000000000001e-05 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38801 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646012+0000", 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3796416750000002, 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030498862000000002, 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.041 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38801 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646012+0000", 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646012+0000", 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646011+0000", 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646019+0000", 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1799999999999999e-06 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646025+0000", 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3269999999999997e-06 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646030+0000", 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676495+0000", 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030465587999999998 2024-09-13T09:26:41.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676511+0000", 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5265e-05 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38802 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:41.045 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3796194750000002, 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030529230000000001, 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38802 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646034+0000", 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646036+0000", 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.435e-06 2024-09-13T09:26:41.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646042+0000", 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5589999999999996e-06 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646058+0000", 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6871000000000002e-05 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676552+0000", 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030493466 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676563+0000", 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1303e-05 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.049 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38803 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3795800890000001, 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030546878, 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38803 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646073+0000", 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646076+0000", 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.557e-06 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646084+0000", 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646088+0000", 2024-09-13T09:26:41.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676606+0000", 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030518749000000001 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676620+0000", 2024-09-13T09:26:41.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3902e-05 2024-09-13T09:26:41.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38804 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3795630299999999, 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030555835999999999, 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38804 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.057 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646090+0000", 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646092+0000", 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7659999999999999e-06 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646095+0000", 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646106+0000", 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1275e-05 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676640+0000", 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030533716999999998 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676646+0000", 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3609999999999997e-06 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38805 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3795401160000003, 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030569447999999999, 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38805 2024-09-13T09:26:41.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.062 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:41.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646113+0000", 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646120+0000", 2024-09-13T09:26:41.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9619999999999996e-06 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646124+0000", 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.335e-06 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646129+0000", 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3830000000000002e-06 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676675+0000", 2024-09-13T09:26:41.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030546923 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676683+0000", 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2729999999999997e-06 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38806 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3795223989999998, 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030602078000000001, 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38806 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646131+0000", 2024-09-13T09:26:41.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646132+0000", 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.522e-06 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646135+0000", 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0079999999999998e-06 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646138+0000", 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676721+0000", 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030583128000000001 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676733+0000", 2024-09-13T09:26:41.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1999e-05 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38807 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3795132240000001, 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622640999999999, 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.070 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38807 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646140+0000", 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646141+0000", 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.381e-06 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646144+0000", 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:26:41.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646147+0000", 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.525e-06 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676752+0000", 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030605591000000001 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676763+0000", 2024-09-13T09:26:41.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0543999999999999e-05 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38912 3.6 3.e4988ec6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.379504539, 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030664937, 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.074 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38912 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646149+0000", 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646150+0000", 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4899999999999999e-06 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646154+0000", 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8449999999999996e-06 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646158+0000", 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.549e-06 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676800+0000", 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030641462000000001 2024-09-13T09:26:41.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676814+0000", 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3844e-05 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38913 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646163+0000", 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3794905970000002, 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030679339, 2024-09-13T09:26:41.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38913 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646163+0000", 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646163+0000", 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646162+0000", 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646165+0000", 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4229999999999999e-06 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646171+0000", 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646176+0000", 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676830+0000", 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030653762000000001 2024-09-13T09:26:41.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676842+0000", 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2211e-05 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38914 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646190+0000", 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3794635150000003, 2024-09-13T09:26:41.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030690914, 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38914 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646190+0000", 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646190+0000", 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646189+0000", 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646192+0000", 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9809999999999999e-06 2024-09-13T09:26:41.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646199+0000", 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9450000000000004e-06 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646204+0000", 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8860000000000003e-06 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676873+0000", 2024-09-13T09:26:41.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030668805 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676881+0000", 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7959999999999996e-06 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38915 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646209+0000", 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3794448629999998, 2024-09-13T09:26:41.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030705501999999999, 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38915 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646209+0000", 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646209+0000", 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646208+0000", 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646211+0000", 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4260000000000002e-06 2024-09-13T09:26:41.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646216+0000", 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7880000000000003e-06 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646221+0000", 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3930000000000001e-06 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676906+0000", 2024-09-13T09:26:41.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030685632000000001 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676914+0000", 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38916 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3794283900000002, 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030739682000000001, 2024-09-13T09:26:41.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38916 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646225+0000", 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646227+0000", 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5210000000000001e-06 2024-09-13T09:26:41.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646233+0000", 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.083e-06 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646238+0000", 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7049999999999998e-06 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676950+0000", 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030711883999999998 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676965+0000", 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4854e-05 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.094 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3794115070000004, 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:26:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.098 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38918 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3793461279999999, 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030720447000000001, 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38918 2024-09-13T09:26:41.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646307+0000", 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646310+0000", 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6490000000000001e-06 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646319+0000", 2024-09-13T09:26:41.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6149999999999997e-06 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646324+0000", 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8250000000000004e-06 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677015+0000", 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030691408 2024-09-13T09:26:41.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677028+0000", 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2527000000000001e-05 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38919 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.103 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3793254429999999, 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030725642000000001, 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38919 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:41.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646328+0000", 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646330+0000", 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4530000000000001e-06 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646337+0000", 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2069999999999998e-06 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646342+0000", 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8559999999999998e-06 2024-09-13T09:26:41.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677041+0000", 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030698578000000001 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677054+0000", 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2896e-05 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38920 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.107 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3793073109999998, 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030735129999999999, 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38920 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646346+0000", 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646348+0000", 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646354+0000", 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646359+0000", 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7469999999999996e-06 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677074+0000", 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030715498000000001 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677081+0000", 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.092e-06 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:41.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3792909450000002, 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:26:41.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 2.906517762, 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:41.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:26:41.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:41.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.121 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:41.123 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:41.123 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:41.115+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:41.224 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:41.220+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:41.289 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:41.289 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:41.289 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:41.289 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:41.300 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:41.324 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:41.320+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:41.418 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:41.419 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:41.419 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:41.419 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:41.419 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:41.425 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:41.424+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:41.429 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:41.526 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:41.524+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:276 3.4 e27/21 3:27927849:::rbd_data.111867511a81.0000000000000026:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.040387314, 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002929961, 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.550 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504324+0000", 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504366+0000", 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2412e-05 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504407+0000", 2024-09-13T09:26:41.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0329e-05 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504427+0000", 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0089999999999999e-05 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504518+0000", 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0801e-05 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504566+0000", 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8745999999999997e-05 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507192+0000", 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00262546 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507256+0000", 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.741774319000001, 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:26:41.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:26:41.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.333598434000002, 2024-09-13T09:26:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:26:41.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:26:41.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.333408310999999, 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:26:41.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.332595902000001, 2024-09-13T09:26:41.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:26:41.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.572 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.143689252999998, 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:41.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:26:41.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4109 3.1f e35/25 3:fb0b19d8:::rbd_data.1118fb4e889d.0000000000000006:head v 35'26, mlcod=31'23)", 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.078618483, 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028372279999999998, 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466094+0000", 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466099+0000", 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7079999999999996e-06 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466120+0000", 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1491000000000001e-05 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466130+0000", 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:41.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466228+0000", 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8072000000000006e-05 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.466258+0000", 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9842e-05 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.468885+0000", 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026277119999999999 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.468932+0000", 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6322000000000001e-05 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:41.582 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.578579867999998, 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:26:41.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.505040855000001, 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:26:41.587 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:26:41.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.412497487, 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:26:41.592 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:26:41.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:26:41.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:24464 3.9 e45/21 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:head v 45'68, mlcod=44'66)", 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.392901760000001, 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028365740000000001, 2024-09-13T09:26:41.597 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151811+0000", 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.151810+0000", 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152495+0000", 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00068434500000000005 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152521+0000", 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6288e-05 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152534+0000", 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152635+0000", 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100758 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.152669+0000", 2024-09-13T09:26:41.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4826999999999997e-05 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.154591+0000", 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001921407 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:23.154648+0000", 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6826000000000001e-05 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.384436911000002, 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:26:41.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.370179128, 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.606 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:26:41.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35667 3.0 e52/21 3:07bd3fb7:::rbd_data.11185dd0f6b.0000000000000020:head v 52'123, mlcod=51'120)", 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.625573+0000", 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9191397079999994, 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024456790000000001, 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625573+0000", 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625573+0000", 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625572+0000", 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625586+0000", 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4744e-05 2024-09-13T09:26:41.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625617+0000", 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0071000000000001e-05 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625631+0000", 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4912000000000001e-05 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625760+0000", 2024-09-13T09:26:41.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012875199999999999 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.625835+0000", 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5018000000000006e-05 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.627958+0000", 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021224590000000002 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.628019+0000", 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1138999999999999e-05 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.9014606749999992, 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.615 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:26:41.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:26:41.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:36222 3.0 e53/21 3:05157b6e:::rbd_object_map.111818d3d5a9:head v 53'124, mlcod=51'121)", 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:33.556903+0000", 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.9878094800000001, 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002616981, 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.556903+0000", 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.556903+0000", 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.556903+0000", 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.556917+0000", 2024-09-13T09:26:41.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3834e-05 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.556945+0000", 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8314e-05 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.556959+0000", 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4389999999999999e-05 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.557096+0000", 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013642299999999999 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.557155+0000", 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9397000000000001e-05 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.559423+0000", 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022679610000000002 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:33.559520+0000", 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7192000000000001e-05 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.9125359820000001, 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.624 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:26:41.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.628 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.707509934, 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:26:41.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.6091718859999999, 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.633 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:26:41.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.637 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.6083632809999999, 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:26:41.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:41.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:41.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:41.642 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:41.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:41.644 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:41.635+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:41.745 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:41.740+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:41.761 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:41.762 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:41.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:41.845 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:41.844+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:41.882 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:41.882 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:41.883 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:41.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:41.946 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:41.943+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:42.002 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:3063 3.9 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.618114927999997, 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031119964999999999, 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 3063 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382213+0000", 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382211+0000", 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382223+0000", 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2245e-05 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382253+0000", 2024-09-13T09:26:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9947999999999999e-05 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382267+0000", 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4034e-05 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382338+0000", 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0931000000000002e-05 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.382439+0000", 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0001009 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.390414+0000", 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079749260000000002 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413261+0000", 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022847375 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413269+0000", 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1170000000000004e-06 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413302+0000", 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3262000000000001e-05 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:05.413333+0000", 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0077999999999999e-05 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:22720 3.1b 3:d9bd9d03:::rbd_data.111811db2be7.00000000000000b7:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.770651289, 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033396362999999998, 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22720 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.009 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229676+0000", 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229675+0000", 2024-09-13T09:26:42.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229684+0000", 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229703+0000", 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9276999999999999e-05 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229713+0000", 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6979999999999993e-06 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229770+0000", 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7096999999999999e-05 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.229856+0000", 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5376000000000004e-05 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.236948+0000", 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070919360000000001 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262783+0000", 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025835137000000001 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262895+0000", 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011265 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.262993+0000", 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8052000000000002e-05 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:19.263073+0000", 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9244000000000002e-05 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25508 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5935104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045792938000002, 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029767913999999999, 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25508 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954535+0000", 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954534+0000", 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954538+0000", 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8010000000000001e-06 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954546+0000", 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5499999999999995e-06 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954553+0000", 2024-09-13T09:26:42.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0049999999999998e-06 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984129+0000", 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029576203999999998 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984202+0000", 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000006e-05 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984303+0000", 2024-09-13T09:26:42.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010026600000000001 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25509 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5963776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045768475999999, 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029860339, 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25509 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954559+0000", 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954562+0000", 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3100000000000001e-06 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954571+0000", 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.816e-06 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954578+0000", 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984320+0000", 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029742527000000001 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984361+0000", 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0615999999999999e-05 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984419+0000", 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8594000000000002e-05 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25510 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 5992448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045743755, 2024-09-13T09:26:42.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030038194000000001, 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25510 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954584+0000", 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954583+0000", 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954587+0000", 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5640000000000001e-06 2024-09-13T09:26:42.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954595+0000", 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.473e-06 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954602+0000", 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9600000000000003e-06 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984458+0000", 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029856186 2024-09-13T09:26:42.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984529+0000", 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0345000000000005e-05 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984622+0000", 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3197999999999998e-05 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25511 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6021120~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045719085000002, 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030144265, 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25511 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.028 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954608+0000", 2024-09-13T09:26:42.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954632+0000", 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3645000000000001e-05 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954641+0000", 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4490000000000005e-06 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954649+0000", 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3750000000000003e-06 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984638+0000", 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029988471999999999 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984678+0000", 2024-09-13T09:26:42.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0170999999999999e-05 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984753+0000", 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4666999999999995e-05 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25512 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6033408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045671443, 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030307508, 2024-09-13T09:26:42.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25512 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954656+0000", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954655+0000", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954659+0000", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8539999999999999e-06 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954668+0000", 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8810000000000002e-06 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954677+0000", 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1530000000000003e-06 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984792+0000", 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030114499999999999 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984866+0000", 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4437000000000006e-05 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984964+0000", 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7330000000000005e-05 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25513 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6062080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045645904000001, 2024-09-13T09:26:42.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030400765999999999, 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25513 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:42.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954682+0000", 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954681+0000", 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954687+0000", 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5039999999999999e-06 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954693+0000", 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0499999999999999e-06 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954700+0000", 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3089999999999998e-06 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.984980+0000", 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280509000000001 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985022+0000", 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1301000000000003e-05 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985082+0000", 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0726000000000001e-05 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25514 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6090752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045621452999999, 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030608730000000001, 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25514 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954706+0000", 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954705+0000", 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954709+0000", 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7290000000000002e-06 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954718+0000", 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8049999999999996e-06 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954725+0000", 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8159999999999996e-06 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985121+0000", 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030396636000000001 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985193+0000", 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1874999999999999e-05 2024-09-13T09:26:42.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985315+0000", 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121452 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25515 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6119424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045596387, 2024-09-13T09:26:42.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030701790999999999, 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25515 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954731+0000", 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954730+0000", 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954734+0000", 2024-09-13T09:26:42.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8449999999999996e-06 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954744+0000", 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0222e-05 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954752+0000", 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2049999999999996e-06 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985334+0000", 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030582024999999999 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985375+0000", 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1253e-05 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985433+0000", 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7986999999999999e-05 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25516 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6148096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045569969999999, 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030869497999999999, 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25516 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954758+0000", 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954757+0000", 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954761+0000", 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7479999999999999e-06 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954803+0000", 2024-09-13T09:26:42.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2342000000000002e-05 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954811+0000", 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0120000000000005e-06 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985473+0000", 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030661702999999998 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985541+0000", 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7800999999999997e-05 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985627+0000", 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6446000000000005e-05 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25517 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6176768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.054 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045508586, 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030924971999999998, 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25517 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954819+0000", 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954818+0000", 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954823+0000", 2024-09-13T09:26:42.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954834+0000", 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1174e-05 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954842+0000", 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0339999999999997e-06 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985645+0000", 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030803298 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985687+0000", 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2089000000000003e-05 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985744+0000", 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6666999999999998e-05 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25518 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6205440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045478919000001, 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031093369999999999, 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25518 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.059 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954849+0000", 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954848+0000", 2024-09-13T09:26:42.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954852+0000", 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8820000000000003e-06 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954860+0000", 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:26:42.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954868+0000", 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6680000000000001e-06 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985784+0000", 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03091648 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985852+0000", 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7407e-05 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985942+0000", 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25519 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6234112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045452624999999, 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031185332, 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25519 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954875+0000", 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954874+0000", 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954878+0000", 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7409999999999998e-06 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954886+0000", 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6209999999999993e-06 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954894+0000", 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3409999999999997e-06 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.985960+0000", 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031066527 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986002+0000", 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1925999999999998e-05 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986060+0000", 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7948999999999997e-05 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25520 3.9 3:93329070:::rbd_data.1118f0c8e51e.0000000000000002:10 [read 6262784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045424689000001, 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031383412999999999, 2024-09-13T09:26:42.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25520 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954903+0000", 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954902+0000", 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954906+0000", 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8060000000000001e-06 2024-09-13T09:26:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954914+0000", 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3059999999999998e-06 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954923+0000", 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7579999999999998e-06 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986106+0000", 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031182399 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986184+0000", 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8699e-05 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986286+0000", 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101977 2024-09-13T09:26:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25563 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.045380470000001, 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034890003000000003, 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25563 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954947+0000", 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954946+0000", 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954953+0000", 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-13T09:26:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954963+0000", 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0468e-05 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.954971+0000", 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2600000000000005e-06 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986090+0000", 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031118930999999999 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986167+0000", 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6470999999999993e-05 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986448+0000", 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000281451 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989625+0000", 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003177352 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989740+0000", 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114632 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989798+0000", 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7555e-05 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.989837+0000", 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9385999999999999e-05 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:25577 3.1 3:874aadaf:::rbd_object_map.111812ca204d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.040459179999999, 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032046094999999997, 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 25577 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959868+0000", 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959867+0000", 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959875+0000", 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9859999999999994e-06 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959888+0000", 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2977e-05 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.959898+0000", 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9280000000000005e-06 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.986894+0000", 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026995473999999998 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.987025+0000", 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013186200000000001 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.987422+0000", 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00039672500000000001 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.990604+0000", 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00318207 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991786+0000", 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001181229 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991869+0000", 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3726000000000004e-05 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:23.991914+0000", 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207000000000003e-05 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:35809 3.3 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 12485461~20431] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:31.818266+0000", 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.182061063000001, 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.051355610000000003, 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35809 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.818266+0000", 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.818266+0000", 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.818265+0000", 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.835941+0000", 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017675535999999999 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.835978+0000", 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6888000000000003e-05 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.835990+0000", 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1960999999999999e-05 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.836072+0000", 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2557000000000003e-05 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.836272+0000", 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000199784 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.837836+0000", 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001564223 2024-09-13T09:26:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.863431+0000", 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025595374000000001 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869497+0000", 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060658170000000003 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869570+0000", 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2679e-05 2024-09-13T09:26:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869622+0000", 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2113000000000001e-05 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:36139 3.3 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:32.431022+0000", 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.569305816, 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039760744000000001, 2024-09-13T09:26:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 36139 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431022+0000", 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431022+0000", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431020+0000", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431032+0000", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.255e-05 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431065+0000", 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3154999999999997e-05 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431080+0000", 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4714e-05 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431154+0000", 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4158000000000004e-05 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.431283+0000", 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012883999999999999 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.440863+0000", 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095793730000000004 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470663+0000", 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029800462999999999 2024-09-13T09:26:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470705+0000", 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2352999999999999e-05 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470744+0000", 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8495000000000003e-05 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:32.470782+0000", 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8618999999999997e-05 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:44939 3.9 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 7556420~45856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.827695+0000", 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1726330239999996, 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037005198000000003, 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 44939 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.827695+0000", 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.827695+0000", 2024-09-13T09:26:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.827693+0000", 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.837414+0000", 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0097209089999999998 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.837438+0000", 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3904999999999999e-05 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.837449+0000", 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.837559+0000", 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011046499999999999 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.837724+0000", 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016413300000000001 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.838057+0000", 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000333987 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.848668+0000", 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010610158 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.864537+0000", 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015869476 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.864647+0000", 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109678 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.864700+0000", 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2899000000000003e-05 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.104 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.105 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:42.106 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:42.099+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:42.206 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:42.204+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:42.220 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:42.221 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:42.307 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:42.303+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:42.340 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:42.341 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:42.408 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:42.403+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:42.460 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.991893646000001, 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.967551383999997, 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.963183020999999, 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.599754363000002, 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.763443882000001, 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:42.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:42.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:42.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.510437853999999, 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:42.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:42.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29819 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:42.495 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.006200715999999, 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022753847000000001, 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29819 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450415+0000", 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450418+0000", 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450423+0000", 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450428+0000", 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463252+0000", 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012823467999999999 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463451+0000", 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019896599999999999 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473170+0000", 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009718829 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29927 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.004584734000002, 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016758252000000001, 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29927 2024-09-13T09:26:42.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452031+0000", 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452034+0000", 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452039+0000", 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.378e-06 2024-09-13T09:26:42.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452043+0000", 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468625+0000", 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581722 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468701+0000", 2024-09-13T09:26:42.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5909999999999997e-05 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468790+0000", 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8980999999999994e-05 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:42981 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.643475+0000", 2024-09-13T09:26:42.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8131416160000002, 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019868784, 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 42981 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.643475+0000", 2024-09-13T09:26:42.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.643475+0000", 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.643473+0000", 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650094+0000", 2024-09-13T09:26:42.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066214239999999999 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650122+0000", 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7912e-05 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650134+0000", 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1426e-05 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650258+0000", 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012432 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650451+0000", 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019276699999999999 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.663170+0000", 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012719398999999999 2024-09-13T09:26:42.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.663241+0000", 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1282e-05 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.663344+0000", 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102348 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46163 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4717971729999997, 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01686329, 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46163 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984821+0000", 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984827+0000", 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6849999999999998e-06 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984833+0000", 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6210000000000001e-06 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001542+0000", 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016709689 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001616+0000", 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3904000000000003e-05 2024-09-13T09:26:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001683+0000", 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6370000000000003e-05 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46164 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2269184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984859+0000", 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4717574930000001, 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016951495, 2024-09-13T09:26:42.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46164 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984859+0000", 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984859+0000", 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984858+0000", 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984862+0000", 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984868+0000", 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2360000000000001e-06 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984873+0000", 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001706+0000", 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016833276000000001 2024-09-13T09:26:42.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001767+0000", 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0563000000000003e-05 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001810+0000", 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3818e-05 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46165 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4717392499999997, 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017006796000000001, 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46165 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984879+0000", 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984885+0000", 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984890+0000", 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.853e-06 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001824+0000", 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016934484 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001857+0000", 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3101e-05 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001884+0000", 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6953000000000001e-05 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46176 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2113536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4717233289999996, 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017134598000000001, 2024-09-13T09:26:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46176 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984895+0000", 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984900+0000", 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3199999999999999e-06 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984905+0000", 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6569999999999996e-06 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001899+0000", 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016993670999999998 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001956+0000", 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6952e-05 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002028+0000", 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2037000000000002e-05 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46178 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2138112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4717074449999998, 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017272180000000002, 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46178 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984917+0000", 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5380000000000002e-06 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984923+0000", 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8039999999999998e-06 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984930+0000", 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9059999999999996e-06 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002051+0000", 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017121474000000001 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002113+0000", 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1329999999999997e-05 2024-09-13T09:26:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002181+0000", 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8474999999999997e-05 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46179 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4716812800000003, 2024-09-13T09:26:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017345030000000001, 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46179 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984938+0000", 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984944+0000", 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984950+0000", 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.789e-06 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002204+0000", 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017253933999999999 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002248+0000", 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000002e-05 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002280+0000", 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2029000000000001e-05 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46180 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2228224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4716616389999997, 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017427570999999999, 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46180 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984957+0000", 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984962+0000", 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3639999999999998e-06 2024-09-13T09:26:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984967+0000", 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8720000000000001e-06 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002300+0000", 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017333047000000001 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002333+0000", 2024-09-13T09:26:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3003999999999998e-05 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002382+0000", 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9206999999999998e-05 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46181 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4716443359999998, 2024-09-13T09:26:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017496483, 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46181 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984975+0000", 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3579999999999999e-06 2024-09-13T09:26:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984981+0000", 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6579999999999999e-06 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984987+0000", 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002398+0000", 2024-09-13T09:26:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017410345000000001 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002439+0000", 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1187999999999999e-05 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002469+0000", 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9774999999999999e-05 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46182 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2252800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984992+0000", 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4716247139999998, 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017589064000000001, 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46182 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984992+0000", 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984992+0000", 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984991+0000", 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984994+0000", 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.441e-06 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984999+0000", 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3070000000000002e-06 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985004+0000", 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002481+0000", 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017477393000000001 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002518+0000", 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7175000000000002e-05 2024-09-13T09:26:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002581+0000", 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46185 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.985008+0000", 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4716087260000004, 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017720248000000001, 2024-09-13T09:26:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46185 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985008+0000", 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985008+0000", 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985007+0000", 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985010+0000", 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3530000000000002e-06 2024-09-13T09:26:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985015+0000", 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159999999999997e-06 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985020+0000", 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002612+0000", 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017592251999999999 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002689+0000", 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6636000000000003e-05 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002728+0000", 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8897999999999999e-05 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46597 3.5 3:a3922975:::rbd_data.111838d2a860.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:38.198483+0000", 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.25813343, 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026482525, 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46597 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198483+0000", 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198483+0000", 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198481+0000", 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198492+0000", 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0746e-05 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198516+0000", 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3839e-05 2024-09-13T09:26:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198527+0000", 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0566e-05 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198624+0000", 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7775999999999994e-05 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198739+0000", 2024-09-13T09:26:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011449 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.224739+0000", 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025999813 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.224846+0000", 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107114 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.224966+0000", 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011984100000000001 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.560 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.560 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:26:42.562 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:42.555+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:42.562 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:42.662 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:42.660+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:42.677 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.677 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:42.677 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:42.677 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:42.763 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:42.755+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:42.789 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.790 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:42.790 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:42.790 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:42.790 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:42.818 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:42.864 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:42.860+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:42.907 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:42.907 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:42.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:42.907 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:42.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.345059208000002, 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:42.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29295 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.462775232999999, 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031712789999999998, 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29295 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.914 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441416+0000", 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441422+0000", 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.592e-06 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441427+0000", 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.453205+0000", 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011778116999999999 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454881+0000", 2024-09-13T09:26:42.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016761879999999999 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473127+0000", 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018246294 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29497 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 819200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459729284000002, 2024-09-13T09:26:42.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024159292999999998, 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29497 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444460+0000", 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444462+0000", 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5129999999999999e-06 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444468+0000", 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4009999999999996e-06 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444475+0000", 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8990000000000001e-06 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.465234+0000", 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020758316999999998 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.465299+0000", 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4873e-05 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468620+0000", 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033208769999999999 2024-09-13T09:26:42.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29498 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459710068, 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024172662000000001, 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29498 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444479+0000", 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444481+0000", 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444486+0000", 2024-09-13T09:26:42.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1070000000000004e-06 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444491+0000", 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2379999999999999e-06 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468645+0000", 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024153813999999999 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468652+0000", 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6290000000000001e-06 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29499 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459695062000002, 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024184213, 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29499 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:42.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444494+0000", 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444496+0000", 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3039999999999999e-06 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444502+0000", 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444506+0000", 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.279e-06 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468672+0000", 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02416565 2024-09-13T09:26:42.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468679+0000", 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9959999999999996e-06 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29500 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459676730999998, 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024191877000000001, 2024-09-13T09:26:42.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29500 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444513+0000", 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444515+0000", 2024-09-13T09:26:42.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1909999999999999e-06 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444520+0000", 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6230000000000002e-06 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444525+0000", 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5040000000000004e-06 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468698+0000", 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024173310999999999 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468705+0000", 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.545e-06 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29501 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459660954, 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024199919, 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29501 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444529+0000", 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444528+0000", 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444531+0000", 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.311e-06 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444536+0000", 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1880000000000001e-06 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444540+0000", 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4009999999999999e-06 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468722+0000", 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024182209 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468729+0000", 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29502 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459645535, 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024207818999999998, 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29502 2024-09-13T09:26:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444544+0000", 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444549+0000", 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4929999999999998e-06 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444554+0000", 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-13T09:26:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444559+0000", 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0980000000000004e-06 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468745+0000", 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024186736 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468752+0000", 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5220000000000004e-06 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29503 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459627059999999, 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024211703000000001, 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29503 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444562+0000", 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444564+0000", 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1500000000000002e-06 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444570+0000", 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4550000000000003e-06 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444574+0000", 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4129999999999999e-06 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468768+0000", 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024193840000000001 2024-09-13T09:26:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468774+0000", 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1700000000000002e-06 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29504 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459611548000002, 2024-09-13T09:26:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024219068999999999, 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29504 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444578+0000", 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444580+0000", 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:26:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444585+0000", 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4260000000000002e-06 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444590+0000", 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2289999999999997e-06 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468791+0000", 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024201139999999999 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468797+0000", 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2960000000000004e-06 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29542 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459596141999999, 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024226968000000002, 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29542 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444593+0000", 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444601+0000", 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5900000000000002e-06 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444607+0000", 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9660000000000001e-06 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444611+0000", 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6920000000000001e-06 2024-09-13T09:26:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468814+0000", 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024202893 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468820+0000", 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1659999999999999e-06 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29544 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459263678999999, 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023918539999999999, 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29544 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444926+0000", 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444925+0000", 2024-09-13T09:26:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444933+0000", 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3640000000000001e-06 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444941+0000", 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9449999999999993e-06 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444947+0000", 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2999999999999998e-06 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468838+0000", 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023891194000000001 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468844+0000", 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3380000000000001e-06 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29545 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459238059, 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023916804, 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29545 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444951+0000", 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444954+0000", 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.615e-06 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444959+0000", 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7169999999999996e-06 2024-09-13T09:26:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444964+0000", 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468862+0000", 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023898189 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468868+0000", 2024-09-13T09:26:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4330000000000001e-06 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29546 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459222134000001, 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023923954000000001, 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29546 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444967+0000", 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444969+0000", 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444975+0000", 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4940000000000003e-06 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444979+0000", 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5379999999999996e-06 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468885+0000", 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023905531000000001 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468891+0000", 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.455e-06 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29547 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459206252000001, 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023930462, 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29547 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444983+0000", 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444985+0000", 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3290000000000001e-06 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444992+0000", 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709999999999996e-06 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.444997+0000", 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3409999999999996e-06 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468907+0000", 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023910579000000001 2024-09-13T09:26:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468914+0000", 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4019999999999999e-06 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29548 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459188859000001, 2024-09-13T09:26:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023936915, 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29548 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445001+0000", 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445000+0000", 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445003+0000", 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3140000000000002e-06 2024-09-13T09:26:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445008+0000", 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445017+0000", 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.533e-06 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468931+0000", 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023913750000000001 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468938+0000", 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29549 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459167593, 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023939946, 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29549 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445022+0000", 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445024+0000", 2024-09-13T09:26:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4930000000000001e-06 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445030+0000", 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.088e-06 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445035+0000", 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468955+0000", 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023920675999999998 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468962+0000", 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29550 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459150883, 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023946261999999999, 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29550 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445039+0000", 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445038+0000", 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445041+0000", 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3329999999999999e-06 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445046+0000", 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445051+0000", 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5149999999999997e-06 2024-09-13T09:26:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468979+0000", 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023927166 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468985+0000", 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29551 3.a 3.348a04aa (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.459119425000001, 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023937653, 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29551 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445070+0000", 2024-09-13T09:26:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445073+0000", 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0070000000000002e-06 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445079+0000", 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6599999999999998e-06 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.445084+0000", 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8169999999999998e-06 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.469002+0000", 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023917559000000001 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.469008+0000", 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1319999999999999e-06 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:35809 3.3 e52/25 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head v 52'133, mlcod=52'130)", 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.065686249000001, 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03073795, 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838501+0000", 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853352+0000", 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014850436999999999 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853379+0000", 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7291e-05 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853393+0000", 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853498+0000", 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010524899999999999 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853523+0000", 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4800999999999999e-05 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869155+0000", 2024-09-13T09:26:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015632732999999999 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869241+0000", 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5935999999999998e-05 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:42.994 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:42.996 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:42.996 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:42.996 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:42.997 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:42.992+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:43.002 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:43.026 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:43.050 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:43.075 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:43.096 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:43.092+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:43.097 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:43.097 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:43.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:43.102 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:43.155 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.156 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:43.209 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:43.208+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:43.273 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.273 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:43.298 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:43.295+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:43.391 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.391 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.400 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:43.395+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:43.401 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:43.501 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:43.496+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:43.521 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.521 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.531 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:43.543 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-13T09:26:43.544 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:43.544 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T09:26:43.544 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T09:26:43.577 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:26:43.601 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:43.600+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:43.643 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone19 intact 2024-09-13T09:26:43.652 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.652 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:43.702 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:43.700+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:43.728 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T09:26:43.785 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.785 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:43.803 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:43.795+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:43.873 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T09:26:43.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T09:26:43.904 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:43.900+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:43.908 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:43.908 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:43.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:44.005 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:44.004+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:44.023 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:44.028 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:44.028 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-09-13T09:26:44.028 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T09:26:44.077 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:44.077 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:44.089 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:44.105 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:44.104+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:44.206 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:44.199+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:44.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T09:26:44.213 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:44.213 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:44.224 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:44.306 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:44.299+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:44.381 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T09:26:44.401 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:44.401 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:44.407 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:44.399+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:44.418 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:44.508 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:44.499+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:44.546 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-13T09:26:44.578 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:44.579 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:44.590 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:44.608 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:44.604+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:44.708 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:44.699+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:44.711 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:44.711 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:44.721 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T09:26:44.722 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:44.810 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:44.808+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:44.829 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:44.829 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:44.832 INFO:teuthology.orchestra.run.smithi063.stdout:2202 write 0x3eece1c thru 0x3ef7518 (0xa6fd bytes) 2024-09-13T09:26:44.838 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:44.910 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:44.903+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:45.012 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:45.008+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:45.012 INFO:teuthology.orchestra.run.smithi063.stdout:2203 writesame 0xa2efc26 thru 0xa2f01b1 data_size 0x5(0x58c bytes) 2024-09-13T09:26:45.017 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:45.017 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:45.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:45.054 INFO:teuthology.orchestra.run.smithi063.stdout:2204 read 0x25db079 thru 0x25e8eae (0xde36 bytes) 2024-09-13T09:26:45.103 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:45.149 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:45.143+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:45.176 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:45.176 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:45.186 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:45.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:45.201 INFO:teuthology.orchestra.run.smithi063.stdout:2206 trunc from 0xbd816c7 to 0x21f0209 2024-09-13T09:26:45.213 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:45.212+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:45.267 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:45.292 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:45.317 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:45.319 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:45.312+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:45.348 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:45.384 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:45.384 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:45.397 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:45.442 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:45.440+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:45.451 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:45.472 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:45.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:45.495 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:45.495 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:45.495 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:45.495 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:45.496 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:45.496 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:45.496 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:45.496 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:45.500 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:45.500 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:45.517 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:45.511+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:45.532 INFO:teuthology.orchestra.run.smithi063.stdout:2207 read 0x55afb7 thru 0x56563e (0xa688 bytes) 2024-09-13T09:26:45.573 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:45.573 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:45.573 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:45.573 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:45.573 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:45.587 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:45.594 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:26:45.645 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:45.639+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:45.718 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:45.716+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2258 3.17 3:ee47c4a0:::rbd_data.1118e3e5e25d.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:45.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.275378215000003, 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017311587, 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2258 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464723+0000", 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464721+0000", 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464731+0000", 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0017000000000001e-05 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464751+0000", 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9602e-05 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464760+0000", 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4720000000000001e-06 2024-09-13T09:26:45.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464829+0000", 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8460999999999994e-05 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.464996+0000", 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016689400000000001 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.469065+0000", 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040689439999999997 2024-09-13T09:26:45.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481759+0000", 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012694735 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.481978+0000", 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021833900000000001 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482007+0000", 2024-09-13T09:26:45.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9496999999999999e-05 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.482034+0000", 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7256000000000001e-05 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2264 3.c 3:3011f8c7:::rbd_object_map.1118e3e5e25d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:45.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.251035952999999, 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026886113999999999, 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2264 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:45.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489065+0000", 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489069+0000", 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6659999999999999e-06 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489077+0000", 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7360000000000001e-06 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.489084+0000", 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491417+0000", 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002333501 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491479+0000", 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2403000000000005e-05 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.491657+0000", 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177894 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.492922+0000", 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001264923 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515885+0000", 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022963041999999999 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515922+0000", 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7051000000000002e-05 2024-09-13T09:26:45.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515951+0000", 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870999999999999e-05 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:2266 3.c 3:3502bea4:::rbd_data.1118e3e5e25d.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.246667590000001, 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022556039999999999, 2024-09-13T09:26:45.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2266 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493433+0000", 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493431+0000", 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493445+0000", 2024-09-13T09:26:45.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3287e-05 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493473+0000", 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8422e-05 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493491+0000", 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7932999999999999e-05 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493537+0000", 2024-09-13T09:26:45.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6418000000000002e-05 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.493678+0000", 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014102799999999999 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.498438+0000", 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047592119999999996 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515863+0000", 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017425139999999999 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515957+0000", 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4578000000000001e-05 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515975+0000", 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7309e-05 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:03.515990+0000", 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4922999999999999e-05 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:4479 3.d e35/21 3:b5484a73:::rbd_data.11189fb5ddd6.0000000000000000:head v 35'15, mlcod=30'13)", 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.883238931999998, 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030053638000000001, 2024-09-13T09:26:45.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856862+0000", 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.856859+0000", 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869904+0000", 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013044843 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869943+0000", 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8939000000000001e-05 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.869959+0000", 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5400000000000002e-05 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870076+0000", 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011784 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.870117+0000", 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0948000000000002e-05 2024-09-13T09:26:45.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886857+0000", 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016739772999999999 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:09.886916+0000", 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8607999999999999e-05 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5801 3.5 3:a6ec39f5:::rbd_data.11185454a1c8.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.046928450999999, 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046790782000000003, 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5801 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693173+0000", 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693171+0000", 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:45.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693183+0000", 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1861999999999999e-05 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693223+0000", 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0776999999999999e-05 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693236+0000", 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2296999999999999e-05 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693305+0000", 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8940000000000001e-05 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.693391+0000", 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6572999999999999e-05 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.703480+0000", 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010088824999999999 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739675+0000", 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036194900000000002 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739794+0000", 2024-09-13T09:26:45.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011950700000000001 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739889+0000", 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4604000000000004e-05 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.739963+0000", 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4308999999999996e-05 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:5905 3.c 3:37353899:::rbd_data.11185454a1c8.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.793922422999998, 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023143013, 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5905 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946179+0000", 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946177+0000", 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946190+0000", 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3003000000000001e-05 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946231+0000", 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1613000000000003e-05 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946248+0000", 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6945000000000001e-05 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946367+0000", 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011882799999999999 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.946439+0000", 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1891000000000007e-05 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.950255+0000", 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00381636 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.968911+0000", 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01865559 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969032+0000", 2024-09-13T09:26:45.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121292 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969192+0000", 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159569 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:12.969322+0000", 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129954 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29819 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 602112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.289685285000001, 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022753847000000001, 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29819 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450416+0000", 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450415+0000", 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450418+0000", 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450423+0000", 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.450428+0000", 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8749999999999999e-06 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463252+0000", 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012823467999999999 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.463451+0000", 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019896599999999999 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473170+0000", 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009718829 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29927 3.1c 3:3f1e3608:::rbd_data.111811db2be7.0000000000000005:f [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.288069303, 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016758252000000001, 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29927 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452032+0000", 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452031+0000", 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452034+0000", 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452039+0000", 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.378e-06 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.452043+0000", 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468625+0000", 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016581722 2024-09-13T09:26:45.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468701+0000", 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5909999999999997e-05 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.468790+0000", 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8980999999999994e-05 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:42981 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.643475+0000", 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.0966261849999999, 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019868784, 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 42981 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.643475+0000", 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.643475+0000", 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.643473+0000", 2024-09-13T09:26:45.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650094+0000", 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066214239999999999 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650122+0000", 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7912e-05 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650134+0000", 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1426e-05 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650258+0000", 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012432 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.650451+0000", 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019276699999999999 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.663170+0000", 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012719398999999999 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.663241+0000", 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1282e-05 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.663344+0000", 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102348 2024-09-13T09:26:45.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46163 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7552817420000002, 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01686329, 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46163 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984819+0000", 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984821+0000", 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984827+0000", 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6849999999999998e-06 2024-09-13T09:26:45.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984833+0000", 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6210000000000001e-06 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001542+0000", 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016709689 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001616+0000", 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3904000000000003e-05 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001683+0000", 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6370000000000003e-05 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46164 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2269184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984859+0000", 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7552420619999998, 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016951495, 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46164 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984859+0000", 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984859+0000", 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984858+0000", 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984862+0000", 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984868+0000", 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2360000000000001e-06 2024-09-13T09:26:45.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984873+0000", 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0980000000000001e-06 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001706+0000", 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016833276000000001 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001767+0000", 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0563000000000003e-05 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001810+0000", 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3818e-05 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46165 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4104192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7552238190000002, 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017006796000000001, 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46165 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984877+0000", 2024-09-13T09:26:45.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984879+0000", 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5359999999999999e-06 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984885+0000", 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984890+0000", 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.853e-06 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001824+0000", 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016934484 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001857+0000", 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3101e-05 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001884+0000", 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6953000000000001e-05 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46176 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2113536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7552078980000001, 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017134598000000001, 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46176 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984893+0000", 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984895+0000", 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984900+0000", 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3199999999999999e-06 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984905+0000", 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6569999999999996e-06 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001899+0000", 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.016993670999999998 2024-09-13T09:26:45.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.001956+0000", 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6952e-05 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002028+0000", 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2037000000000002e-05 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46178 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2138112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7551920140000004, 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017272180000000002, 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46178 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984909+0000", 2024-09-13T09:26:45.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984917+0000", 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5380000000000002e-06 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984923+0000", 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8039999999999998e-06 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984930+0000", 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9059999999999996e-06 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002051+0000", 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017121474000000001 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002113+0000", 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1329999999999997e-05 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002181+0000", 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8474999999999997e-05 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46179 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7551658489999999, 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017345030000000001, 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46179 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:45.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984935+0000", 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984938+0000", 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7039999999999999e-06 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984944+0000", 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984950+0000", 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.789e-06 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002204+0000", 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017253933999999999 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002248+0000", 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000002e-05 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002280+0000", 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2029000000000001e-05 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.819 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46180 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2228224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7551462080000002, 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017427570999999999, 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46180 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984955+0000", 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984957+0000", 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984962+0000", 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3639999999999998e-06 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984967+0000", 2024-09-13T09:26:45.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8720000000000001e-06 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002300+0000", 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017333047000000001 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002333+0000", 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3003999999999998e-05 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002382+0000", 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9206999999999998e-05 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46181 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7551289050000003, 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017496483, 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46181 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:45.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984972+0000", 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984975+0000", 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3579999999999999e-06 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984981+0000", 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6579999999999999e-06 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984987+0000", 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002398+0000", 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017410345000000001 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002439+0000", 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1187999999999999e-05 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002469+0000", 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9774999999999999e-05 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46182 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 2252800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.984992+0000", 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7551092830000004, 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017589064000000001, 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46182 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984992+0000", 2024-09-13T09:26:45.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984992+0000", 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984991+0000", 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984994+0000", 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.441e-06 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.984999+0000", 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3070000000000002e-06 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985004+0000", 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002481+0000", 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017477393000000001 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002518+0000", 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7175000000000002e-05 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002581+0000", 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2442999999999998e-05 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46185 3.16 3:6efa2950:::rbd_data.111818d3d5a9.0000000000000009:15 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:37.985008+0000", 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.755093295, 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017720248000000001, 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46185 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985008+0000", 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985008+0000", 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985007+0000", 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985010+0000", 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3530000000000002e-06 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985015+0000", 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159999999999997e-06 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:37.985020+0000", 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002612+0000", 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017592251999999999 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002689+0000", 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6636000000000003e-05 2024-09-13T09:26:45.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.002728+0000", 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8897999999999999e-05 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:46597 3.5 3:a3922975:::rbd_data.111838d2a860.0000000000000014:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:38.198483+0000", 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.5416179989999996, 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026482525, 2024-09-13T09:26:45.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 46597 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198483+0000", 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:45.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198483+0000", 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198481+0000", 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198492+0000", 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0746e-05 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198516+0000", 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3839e-05 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198527+0000", 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0566e-05 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198624+0000", 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7775999999999994e-05 2024-09-13T09:26:45.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.198739+0000", 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011449 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.224739+0000", 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025999813 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.224846+0000", 2024-09-13T09:26:45.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107114 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:38.224966+0000", 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011984100000000001 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:45.842 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:45.843 INFO:teuthology.orchestra.run.smithi063.stdout:2208 writesame 0xac89244 thru 0xac8d942 data_size 0x5(0x46ff bytes) 2024-09-13T09:26:45.844 INFO:teuthology.orchestra.run.smithi063.stdout:2210 writesame 0x7969d34 thru 0x7975c9c data_size 0x1(0xbf69 bytes) 2024-09-13T09:26:45.844 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:45.835+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:45.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:45.863 INFO:teuthology.orchestra.run.smithi063.stdout:2211 punch from 0x9e16f06 to 0x9e26952, (0xfa4c bytes) 2024-09-13T09:26:45.863 INFO:teuthology.orchestra.run.smithi063.stdout:2213 read 0xa392c3d thru 0xa39f4fc (0xc8c0 bytes) 2024-09-13T09:26:45.863 INFO:teuthology.orchestra.run.smithi063.stdout:2214 write 0x1bb2b43 thru 0x1bb4c00 (0x20be bytes) 2024-09-13T09:26:45.944 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:45.940+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:45.998 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:45.998 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:45.998 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:45.998 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:46.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:46.020 INFO:teuthology.orchestra.run.smithi063.stdout:2215 read 0x4314be2 thru 0x4324402 (0xf821 bytes) 2024-09-13T09:26:46.021 INFO:teuthology.orchestra.run.smithi063.stdout:2216 read 0x5f2c69d thru 0x5f3b137 (0xea9b bytes) 2024-09-13T09:26:46.021 INFO:teuthology.orchestra.run.smithi063.stdout:2217 read 0x4d198c9 thru 0x4d2735d (0xda95 bytes) 2024-09-13T09:26:46.021 INFO:teuthology.orchestra.run.smithi063.stdout:2218 read 0x3acfce6 thru 0x3ad454b (0x4866 bytes) 2024-09-13T09:26:46.021 INFO:teuthology.orchestra.run.smithi063.stdout:2220 read 0x110fede thru 0x1111892 (0x19b5 bytes) 2024-09-13T09:26:46.044 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:46.035+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:46.083 INFO:teuthology.orchestra.run.smithi063.stdout:2221 trunc from 0xac8d943 to 0x8bd70f9 2024-09-13T09:26:46.146 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:46.144+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:46.155 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:46.155 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:46.155 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:46.156 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:46.156 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:46.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:46.170 INFO:teuthology.orchestra.run.smithi063.stdout:2222 write 0x42f7166 thru 0x4306fd7 (0xfe72 bytes) 2024-09-13T09:26:46.196 INFO:teuthology.orchestra.run.smithi063.stdout:2224 punch from 0x6270c40 to 0x6274630, (0x39f0 bytes) 2024-09-13T09:26:46.197 INFO:teuthology.orchestra.run.smithi063.stdout:2225 write 0x5f29adf thru 0x5f37a0b (0xdf2d bytes) 2024-09-13T09:26:46.221 INFO:teuthology.orchestra.run.smithi063.stdout:2226 read 0x585a149 thru 0x5860fa6 (0x6e5e bytes) 2024-09-13T09:26:46.224 INFO:teuthology.orchestra.run.smithi063.stdout:2227 trunc from 0x8bd70f9 to 0x9f705f9 2024-09-13T09:26:46.234 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:46.234 INFO:teuthology.orchestra.run.smithi063.stdout:2233 read 0x8fb1f04 thru 0x8fb9d13 (0x7e10 bytes) 2024-09-13T09:26:46.234 INFO:teuthology.orchestra.run.smithi063.stdout:2234 punch from 0x89691a7 to 0x896b047, (0x1ea0 bytes) 2024-09-13T09:26:46.234 INFO:teuthology.orchestra.run.smithi063.stdout:2236 read 0x8208ce8 thru 0x820ec77 (0x5f90 bytes) 2024-09-13T09:26:46.235 INFO:teuthology.orchestra.run.smithi063.stdout:2240 read 0x4fac268 thru 0x4fad3b4 (0x114d bytes) 2024-09-13T09:26:46.235 INFO:teuthology.orchestra.run.smithi063.stdout:2241 read 0x1158f9e thru 0x115ef10 (0x5f73 bytes) 2024-09-13T09:26:46.247 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:46.244+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:46.292 INFO:teuthology.orchestra.run.smithi063.stdout:2243 write 0x6b57259 thru 0x6b5c3c6 (0x516e bytes) 2024-09-13T09:26:46.327 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:46.327 INFO:teuthology.orchestra.run.smithi063.stdout:2246 write 0xebd4ea thru 0xec2b0a (0x5621 bytes) 2024-09-13T09:26:46.348 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:46.344+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:46.371 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.808616479000001, 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:46.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:46.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:46.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:29295 3.a 3:5520512c:::rbd_data.111811db2be7.0000000000000000:f [read 12288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.926332504000001, 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031712789999999998, 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29295 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441414+0000", 2024-09-13T09:26:46.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441416+0000", 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441422+0000", 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.592e-06 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.441427+0000", 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.453205+0000", 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011778116999999999 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.454881+0000", 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016761879999999999 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:24.473127+0000", 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018246294 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:35809 3.3 e52/25 3:c7e33bad:::rbd_data.11184f2468.0000000000000000:head v 52'133, mlcod=52'130)", 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.52924352, 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03073795, 2024-09-13T09:26:46.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838503+0000", 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.838501+0000", 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853352+0000", 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014850436999999999 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853379+0000", 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7291e-05 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853393+0000", 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853498+0000", 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010524899999999999 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.853523+0000", 2024-09-13T09:26:46.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4800999999999999e-05 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869155+0000", 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015632732999999999 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:31.869241+0000", 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5935999999999998e-05 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80120 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1513472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.337939+0000", 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029807813999999998, 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024657794, 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80120 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337939+0000", 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337939+0000", 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337939+0000", 2024-09-13T09:26:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337941+0000", 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337948+0000", 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0269999999999998e-06 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337953+0000", 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0669999999999999e-06 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.362352+0000", 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024398749000000001 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.362445+0000", 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2770999999999997e-05 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.362597+0000", 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000151953 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80121 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1528832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.337958+0000", 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029788816999999999, 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025313883999999998, 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80121 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337958+0000", 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337958+0000", 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337958+0000", 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337960+0000", 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.548e-06 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337967+0000", 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9720000000000003e-06 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337972+0000", 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9420000000000003e-06 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.362622+0000", 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024650002000000001 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.362712+0000", 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9975999999999994e-05 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363272+0000", 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000559776 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80122 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1544192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.337977+0000", 2024-09-13T09:26:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029770264000000001, 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025576700000000001, 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80122 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337977+0000", 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337977+0000", 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337976+0000", 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337979+0000", 2024-09-13T09:26:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.306e-06 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337984+0000", 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5169999999999999e-06 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337989+0000", 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8760000000000004e-06 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363299+0000", 2024-09-13T09:26:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025309767 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363367+0000", 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7935000000000006e-05 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363553+0000", 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000186646 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80123 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1559552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.337993+0000", 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029753835999999999, 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025828285999999999, 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80123 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337993+0000", 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337993+0000", 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337993+0000", 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.337995+0000", 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2639999999999998e-06 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.338002+0000", 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929999999999998e-06 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.338007+0000", 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2159999999999997e-06 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363579+0000", 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025572302000000002 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363646+0000", 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6339000000000003e-05 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363821+0000", 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000175487 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80346 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1053696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339218+0000", 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028528321999999998, 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024846521, 2024-09-13T09:26:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80346 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339218+0000", 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339218+0000", 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339218+0000", 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339222+0000", 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7210000000000002e-06 2024-09-13T09:26:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339230+0000", 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9060000000000002e-06 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339237+0000", 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363846+0000", 2024-09-13T09:26:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024608962000000002 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.363910+0000", 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.436e-05 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364065+0000", 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015470499999999999 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80347 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1069056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339243+0000", 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028503516999999999, 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025096299999999998, 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80347 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339243+0000", 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339243+0000", 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339243+0000", 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339246+0000", 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7319999999999998e-06 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339253+0000", 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1400000000000002e-06 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339258+0000", 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364091+0000", 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024832916999999999 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364163+0000", 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1687000000000007e-05 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364340+0000", 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017703200000000001 2024-09-13T09:26:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80348 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1084416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339262+0000", 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028484349999999999, 2024-09-13T09:26:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025354692000000002, 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80348 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339262+0000", 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339262+0000", 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339262+0000", 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339265+0000", 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6129999999999998e-06 2024-09-13T09:26:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339272+0000", 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0559999999999999e-06 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339277+0000", 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0839999999999999e-06 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364365+0000", 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025088356999999999 2024-09-13T09:26:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364432+0000", 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7216000000000002e-05 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364617+0000", 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018469600000000001 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80349 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1099776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339281+0000", 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028465323000000001, 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025607857000000001, 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80349 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339281+0000", 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339281+0000", 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339281+0000", 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339284+0000", 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339291+0000", 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339296+0000", 2024-09-13T09:26:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364643+0000", 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025347613000000001 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364712+0000", 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8792000000000001e-05 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364889+0000", 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000177381 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80350 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1115136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339300+0000", 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028447223000000001, 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025832536, 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80350 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339300+0000", 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339300+0000", 2024-09-13T09:26:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339299+0000", 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339302+0000", 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6869999999999999e-06 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339307+0000", 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4240000000000001e-06 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339312+0000", 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364914+0000", 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025602377999999999 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.364983+0000", 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8516000000000006e-05 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365132+0000", 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014912599999999999 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.426 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80351 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1130496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339316+0000", 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028430733, 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026101708000000001, 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80351 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339316+0000", 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339316+0000", 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339316+0000", 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339318+0000", 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339326+0000", 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7880000000000007e-06 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339331+0000", 2024-09-13T09:26:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.758e-06 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365157+0000", 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025825931 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365247+0000", 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0334999999999995e-05 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365418+0000", 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017084500000000001 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80352 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1145856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339335+0000", 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.02841204, 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0263533, 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80352 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339335+0000", 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339335+0000", 2024-09-13T09:26:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339334+0000", 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339337+0000", 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9579999999999999e-06 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339343+0000", 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5779999999999998e-06 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339348+0000", 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6820000000000002e-06 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365445+0000", 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026097210999999999 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365514+0000", 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8820999999999997e-05 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365688+0000", 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017444100000000001 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1161216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339352+0000", 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028395041999999999, 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026642729, 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80353 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339352+0000", 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339352+0000", 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339351+0000", 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339354+0000", 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6290000000000002e-06 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339361+0000", 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5050000000000001e-06 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339366+0000", 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9479999999999999e-06 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365714+0000", 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026347265000000002 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365786+0000", 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1977000000000006e-05 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.365995+0000", 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00020881799999999999 2024-09-13T09:26:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80354 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1176576~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339371+0000", 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028376083, 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026867102, 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80354 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339371+0000", 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339371+0000", 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339370+0000", 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339373+0000", 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339380+0000", 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1629999999999999e-06 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339385+0000", 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.921e-06 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366021+0000", 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026635473999999999 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366087+0000", 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6934999999999995e-05 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366238+0000", 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015039400000000001 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1191936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339389+0000", 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028357502999999999, 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027112535, 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80355 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339389+0000", 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339389+0000", 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339389+0000", 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339391+0000", 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4040000000000002e-06 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339397+0000", 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2700000000000004e-06 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339401+0000", 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7849999999999999e-06 2024-09-13T09:26:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366264+0000", 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.026862194999999998 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366332+0000", 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8087e-05 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366502+0000", 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017017200000000001 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80356 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1207296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339405+0000", 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028341410000000001, 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027390172000000001, 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80356 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339405+0000", 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339405+0000", 2024-09-13T09:26:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339405+0000", 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339407+0000", 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3929999999999998e-06 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339415+0000", 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3259999999999998e-06 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339420+0000", 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1220000000000003e-06 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366528+0000", 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02710827 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366596+0000", 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7668000000000003e-05 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366796+0000", 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019972600000000001 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80357 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1222656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339424+0000", 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028322481, 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027652993000000001, 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80357 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339424+0000", 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339424+0000", 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339424+0000", 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339427+0000", 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339433+0000", 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8669999999999996e-06 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339438+0000", 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9150000000000004e-06 2024-09-13T09:26:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366822+0000", 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027383379999999999 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.366890+0000", 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8580999999999999e-05 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.367077+0000", 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018705800000000001 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80358 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1238016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.339443+0000", 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028304241000000001, 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027808506, 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80358 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339443+0000", 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339443+0000", 2024-09-13T09:26:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339442+0000", 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339449+0000", 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6050000000000003e-06 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339454+0000", 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.609e-06 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.339460+0000", 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4670000000000003e-06 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.367121+0000", 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027660844 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.367175+0000", 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4407000000000003e-05 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.367251+0000", 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5904999999999999e-05 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.463 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:46.464 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:46.465 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:46.455+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:46.565 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:46.559+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:46.577 INFO:teuthology.orchestra.run.smithi063.stdout:2248 punch from 0x1c1a422 to 0x1c1b79d, (0x137b bytes) 2024-09-13T09:26:46.588 INFO:teuthology.orchestra.run.smithi063.stdout:2250 trunc from 0x9f705f9 to 0xe64c65c 2024-09-13T09:26:46.601 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:26:46.606 INFO:teuthology.orchestra.run.smithi063.stdout:2252 punch from 0x10720 to 0x1b3f4, (0xacd4 bytes) 2024-09-13T09:26:46.606 INFO:teuthology.orchestra.run.smithi063.stdout:2253 punch from 0xd1bf217 to 0xd1cc287, (0xd070 bytes) 2024-09-13T09:26:46.606 INFO:teuthology.orchestra.run.smithi063.stdout:2254 writesame 0xade3477 thru 0xade365f data_size 0x1(0x1e9 bytes) 2024-09-13T09:26:46.613 INFO:teuthology.orchestra.run.smithi063.stdout:2256 read 0x8bdae8 thru 0x8c9063 (0xb57c bytes) 2024-09-13T09:26:46.668 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:46.664+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84399 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633177+0000", 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028551449, 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028576260999999999, 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84399 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633177+0000", 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633177+0000", 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633177+0000", 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633179+0000", 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2730000000000001e-06 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633184+0000", 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633189+0000", 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1540000000000004e-06 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84531 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644738+0000", 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016990473999999998, 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017088268, 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84531 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644738+0000", 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644738+0000", 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644738+0000", 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644740+0000", 2024-09-13T09:26:46.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644746+0000", 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644750+0000", 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4120000000000003e-06 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84564 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647491+0000", 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014238262999999999, 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014387901999999999, 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84564 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647491+0000", 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647491+0000", 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647490+0000", 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647492+0000", 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2220000000000001e-06 2024-09-13T09:26:46.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647498+0000", 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4280000000000004e-06 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647503+0000", 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.729e-06 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84903 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655703+0000", 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0060259229999999999, 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0062262289999999998, 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84903 2024-09-13T09:26:46.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655703+0000", 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655703+0000", 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655703+0000", 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655705+0000", 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2180000000000002e-06 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655710+0000", 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4049999999999999e-06 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655715+0000", 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84938 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656271+0000", 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054577330000000002, 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057198980000000002, 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84938 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656271+0000", 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656271+0000", 2024-09-13T09:26:46.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656271+0000", 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656273+0000", 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656279+0000", 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656284+0000", 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4349999999999996e-06 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84400 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633193+0000", 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028536021000000002, 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028849687999999998, 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84400 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633193+0000", 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633193+0000", 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633192+0000", 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633201+0000", 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.833e-06 2024-09-13T09:26:46.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633207+0000", 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.694e-06 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633211+0000", 2024-09-13T09:26:46.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84532 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644754+0000", 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016974426000000001, 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017337412, 2024-09-13T09:26:46.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84532 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644754+0000", 2024-09-13T09:26:46.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644754+0000", 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644754+0000", 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644756+0000", 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.345e-06 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644762+0000", 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2519999999999999e-06 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644766+0000", 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2679999999999996e-06 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84566 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647509+0000", 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014220003, 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014640465, 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84566 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647509+0000", 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647509+0000", 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647508+0000", 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647511+0000", 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.491e-06 2024-09-13T09:26:46.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647516+0000", 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647521+0000", 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5619999999999997e-06 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84904 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655719+0000", 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0060099319999999999, 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064800489999999999, 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84904 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655719+0000", 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655719+0000", 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655718+0000", 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655721+0000", 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655726+0000", 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655730+0000", 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4530000000000004e-06 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84940 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656288+0000", 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054403969999999996, 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059620819999999996, 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84940 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656288+0000", 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656288+0000", 2024-09-13T09:26:46.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656288+0000", 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656290+0000", 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4700000000000001e-06 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656296+0000", 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4330000000000003e-06 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656301+0000", 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7759999999999997e-06 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84401 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633215+0000", 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028513525000000001, 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029084387, 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84401 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633215+0000", 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633215+0000", 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633215+0000", 2024-09-13T09:26:46.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633217+0000", 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2960000000000001e-06 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633222+0000", 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.33e-06 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633225+0000", 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6880000000000001e-06 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84533 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644770+0000", 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016959134000000001, 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017579002, 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84533 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644770+0000", 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644770+0000", 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644769+0000", 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644778+0000", 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3890000000000005e-06 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644783+0000", 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.558e-06 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644788+0000", 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84869 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.653816+0000", 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0079129820000000007, 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085811440000000006, 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84869 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.653816+0000", 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.653816+0000", 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.653815+0000", 2024-09-13T09:26:46.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.653821+0000", 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5300000000000002e-06 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.653832+0000", 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0586e-05 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.653840+0000", 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2390000000000002e-06 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84905 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655735+0000", 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059940699999999998, 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067112109999999999, 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84905 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655735+0000", 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655735+0000", 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655734+0000", 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655737+0000", 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4389999999999999e-06 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655742+0000", 2024-09-13T09:26:46.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5609999999999998e-06 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655747+0000", 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8810000000000004e-06 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84942 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656305+0000", 2024-09-13T09:26:46.729 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054242300000000004, 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061902370000000003, 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84942 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656305+0000", 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656305+0000", 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656304+0000", 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656306+0000", 2024-09-13T09:26:46.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2730000000000001e-06 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656312+0000", 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.49e-06 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656316+0000", 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4429999999999996e-06 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84402 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633229+0000", 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028500157000000002, 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029314874000000001, 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84402 2024-09-13T09:26:46.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633229+0000", 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633229+0000", 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633228+0000", 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633235+0000", 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5710000000000003e-06 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633240+0000", 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9139999999999999e-06 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633244+0000", 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8970000000000001e-06 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84534 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644792+0000", 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016936916999999999, 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017800992000000002, 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84534 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644792+0000", 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644792+0000", 2024-09-13T09:26:46.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644792+0000", 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644794+0000", 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4399999999999999e-06 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644799+0000", 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644804+0000", 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4599999999999996e-06 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84871 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.739 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654137+0000", 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0075914820000000001, 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085044690000000006, 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84871 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654137+0000", 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654137+0000", 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654136+0000", 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654142+0000", 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6300000000000003e-06 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654158+0000", 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.624e-05 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654165+0000", 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0319999999999998e-06 2024-09-13T09:26:46.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84906 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655751+0000", 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059773079999999998, 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069441470000000003, 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84906 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655751+0000", 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655751+0000", 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655751+0000", 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655754+0000", 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.621e-06 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655759+0000", 2024-09-13T09:26:46.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2809999999999999e-06 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655764+0000", 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7350000000000004e-06 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84944 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.746 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656321+0000", 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054080639999999998, 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064238630000000001, 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84944 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656321+0000", 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656321+0000", 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656320+0000", 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656323+0000", 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2699999999999999e-06 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656330+0000", 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.08e-06 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656335+0000", 2024-09-13T09:26:46.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4070000000000001e-06 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84403 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633247+0000", 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028481490000000002, 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029545987999999999, 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84403 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633247+0000", 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633247+0000", 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633247+0000", 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633249+0000", 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633253+0000", 2024-09-13T09:26:46.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2930000000000002e-06 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633257+0000", 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8510000000000001e-06 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84535 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644808+0000", 2024-09-13T09:26:46.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016921172000000002, 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01803368, 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84535 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644808+0000", 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644808+0000", 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644807+0000", 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644810+0000", 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3530000000000002e-06 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644815+0000", 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.361e-06 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644819+0000", 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3229999999999999e-06 2024-09-13T09:26:46.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84873 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654183+0000", 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0075452990000000001, 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087065270000000004, 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84873 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654183+0000", 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654183+0000", 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654183+0000", 2024-09-13T09:26:46.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654187+0000", 2024-09-13T09:26:46.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-13T09:26:46.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654194+0000", 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8990000000000004e-06 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654199+0000", 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4070000000000001e-06 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84907 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655768+0000", 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059610899999999996, 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071709549999999997, 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84907 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655768+0000", 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655768+0000", 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655767+0000", 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655770+0000", 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3209999999999999e-06 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655775+0000", 2024-09-13T09:26:46.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1850000000000003e-06 2024-09-13T09:26:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655779+0000", 2024-09-13T09:26:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4120000000000003e-06 2024-09-13T09:26:46.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84946 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656339+0000", 2024-09-13T09:26:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0053897720000000001, 2024-09-13T09:26:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066485490000000001, 2024-09-13T09:26:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84946 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656339+0000", 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656339+0000", 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656339+0000", 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656341+0000", 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656347+0000", 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.604e-06 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656352+0000", 2024-09-13T09:26:46.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7639999999999996e-06 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84404 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633261+0000", 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028467515999999998, 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029774777999999998, 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84404 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633261+0000", 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633261+0000", 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633261+0000", 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633263+0000", 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633268+0000", 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633271+0000", 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9190000000000001e-06 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84536 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644823+0000", 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016905582999999998, 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018262269000000001, 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84536 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644823+0000", 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644823+0000", 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644823+0000", 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644825+0000", 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.311e-06 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644830+0000", 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2859999999999999e-06 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644835+0000", 2024-09-13T09:26:46.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84874 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654206+0000", 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0075228769999999999, 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0089281680000000002, 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84874 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654206+0000", 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654206+0000", 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654205+0000", 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654208+0000", 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654214+0000", 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654218+0000", 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3880000000000002e-06 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84908 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655783+0000", 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059454449999999997, 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007399587, 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84908 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655783+0000", 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655783+0000", 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655783+0000", 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655785+0000", 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3630000000000001e-06 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655792+0000", 2024-09-13T09:26:46.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655796+0000", 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9779999999999996e-06 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84948 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656356+0000", 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0053723060000000003, 2024-09-13T09:26:46.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068755359999999998, 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84948 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656356+0000", 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656356+0000", 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656356+0000", 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656358+0000", 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.306e-06 2024-09-13T09:26:46.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656364+0000", 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3720000000000004e-06 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656368+0000", 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3699999999999997e-06 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84405 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633275+0000", 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028453281, 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030005104000000001, 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84405 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633275+0000", 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633275+0000", 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633275+0000", 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633277+0000", 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9230000000000001e-06 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633282+0000", 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5349999999999998e-06 2024-09-13T09:26:46.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633286+0000", 2024-09-13T09:26:46.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9870000000000001e-06 2024-09-13T09:26:46.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84537 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644839+0000", 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016889682999999999, 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018490335, 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84537 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644839+0000", 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644839+0000", 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644839+0000", 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644841+0000", 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3630000000000001e-06 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644846+0000", 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:26:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644850+0000", 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3189999999999996e-06 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84876 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654222+0000", 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.007506759, 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0091557730000000007, 2024-09-13T09:26:46.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84876 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654222+0000", 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654222+0000", 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654222+0000", 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654224+0000", 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5849999999999998e-06 2024-09-13T09:26:46.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654230+0000", 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5450000000000003e-06 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654234+0000", 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1849999999999997e-06 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84909 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655801+0000", 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059277760000000001, 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0076258999999999997, 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84909 2024-09-13T09:26:46.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655801+0000", 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655801+0000", 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655800+0000", 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655803+0000", 2024-09-13T09:26:46.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9849999999999998e-06 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655809+0000", 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3900000000000001e-06 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655813+0000", 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6519999999999997e-06 2024-09-13T09:26:46.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85000 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.658594+0000", 2024-09-13T09:26:46.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003134927, 2024-09-13T09:26:46.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0048808599999999999, 2024-09-13T09:26:46.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85000 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.658594+0000", 2024-09-13T09:26:46.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.658594+0000", 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.658593+0000", 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.658598+0000", 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8810000000000004e-06 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.658606+0000", 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9670000000000001e-06 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.658612+0000", 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5549999999999999e-06 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84406 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633289+0000", 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028439335, 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030264075000000001, 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84406 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633289+0000", 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633289+0000", 2024-09-13T09:26:46.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633289+0000", 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633291+0000", 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8980000000000001e-06 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633296+0000", 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7430000000000002e-06 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633300+0000", 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7280000000000001e-06 2024-09-13T09:26:46.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84538 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644855+0000", 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016874216000000001, 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018748854999999998, 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84538 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644855+0000", 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644855+0000", 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644854+0000", 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644857+0000", 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.689e-06 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644863+0000", 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644867+0000", 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1749999999999998e-06 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84877 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654240+0000", 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0074883190000000002, 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0094180270000000007, 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84877 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654240+0000", 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654240+0000", 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654240+0000", 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654243+0000", 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6419999999999999e-06 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654248+0000", 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654253+0000", 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3980000000000001e-06 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84910 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655825+0000", 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059033619999999997, 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0078828289999999992, 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84910 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655825+0000", 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655825+0000", 2024-09-13T09:26:46.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655825+0000", 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655828+0000", 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8849999999999999e-06 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655834+0000", 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.998e-06 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655839+0000", 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.045e-06 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85002 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.659887+0000", 2024-09-13T09:26:46.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0018416229999999999, 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038724430000000002, 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85002 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.659887+0000", 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.659887+0000", 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.659886+0000", 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.659892+0000", 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4029999999999998e-06 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662115+0000", 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002223315 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662125+0000", 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9129999999999998e-06 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84407 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633303+0000", 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028425514999999998, 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030511252999999999, 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84407 2024-09-13T09:26:46.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633303+0000", 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633303+0000", 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633303+0000", 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633305+0000", 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633310+0000", 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2950000000000001e-06 2024-09-13T09:26:46.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633315+0000", 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2039999999999999e-06 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84539 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644871+0000", 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016858064999999998, 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018993594999999999, 2024-09-13T09:26:46.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84539 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644871+0000", 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644871+0000", 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644870+0000", 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644873+0000", 2024-09-13T09:26:46.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.655e-06 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644879+0000", 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6690000000000003e-06 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644884+0000", 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8110000000000002e-06 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84879 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654257+0000", 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0074716909999999999, 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096590100000000009, 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84879 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654257+0000", 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654257+0000", 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654257+0000", 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654259+0000", 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8059999999999999e-06 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654265+0000", 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5339999999999999e-06 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654269+0000", 2024-09-13T09:26:46.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.408e-06 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84911 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655844+0000", 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.005885197, 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081221990000000001, 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84911 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655844+0000", 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655844+0000", 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655843+0000", 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655846+0000", 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6010000000000002e-06 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655852+0000", 2024-09-13T09:26:46.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1120000000000001e-06 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655857+0000", 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1070000000000004e-06 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85004 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662134+0000", 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9995952, 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001882328, 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85004 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662134+0000", 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662134+0000", 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662133+0000", 2024-09-13T09:26:46.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662138+0000", 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662146+0000", 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3359999999999995e-06 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662152+0000", 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5820000000000001e-06 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84408 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633319+0000", 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028409885999999999, 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030752824000000002, 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84408 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633319+0000", 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633319+0000", 2024-09-13T09:26:46.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633319+0000", 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633321+0000", 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633326+0000", 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0669999999999999e-06 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633330+0000", 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3050000000000003e-06 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84540 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644888+0000", 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016841212000000001, 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019235924000000001, 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84540 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644888+0000", 2024-09-13T09:26:46.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644888+0000", 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644887+0000", 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644889+0000", 2024-09-13T09:26:46.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.283e-06 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644895+0000", 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2839999999999997e-06 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644899+0000", 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4290000000000003e-06 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84880 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654274+0000", 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0074550950000000001, 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098983460000000006, 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84880 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654274+0000", 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654274+0000", 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654273+0000", 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654276+0000", 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2129999999999998e-06 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654281+0000", 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2610000000000001e-06 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654285+0000", 2024-09-13T09:26:46.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3560000000000003e-06 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84912 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655861+0000", 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0058673550000000003, 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083619039999999999, 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84912 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655861+0000", 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655861+0000", 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655861+0000", 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655863+0000", 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3949999999999999e-06 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655869+0000", 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6559999999999997e-06 2024-09-13T09:26:46.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655874+0000", 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85006 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662156+0000", 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9995728, 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021172970000000002, 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85006 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662156+0000", 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662156+0000", 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662155+0000", 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662158+0000", 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.965e-06 2024-09-13T09:26:46.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662164+0000", 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8409999999999996e-06 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662169+0000", 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84409 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633334+0000", 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028394510000000001, 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030994183000000002, 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84409 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633334+0000", 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633334+0000", 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633334+0000", 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633336+0000", 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633342+0000", 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3210000000000004e-06 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633346+0000", 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4530000000000004e-06 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84541 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644903+0000", 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016825908000000001, 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019474422000000002, 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84541 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644903+0000", 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644903+0000", 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644903+0000", 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644905+0000", 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644911+0000", 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6740000000000002e-06 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644915+0000", 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4950000000000002e-06 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84881 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654292+0000", 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.007437239, 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010136875, 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84881 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654292+0000", 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654292+0000", 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654291+0000", 2024-09-13T09:26:46.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654294+0000", 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654299+0000", 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2719999999999997e-06 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654303+0000", 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2849999999999996e-06 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84913 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655880+0000", 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0058489809999999996, 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085976999999999998, 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84913 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655880+0000", 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655880+0000", 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655879+0000", 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655882+0000", 2024-09-13T09:26:46.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655889+0000", 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0659999999999998e-06 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655895+0000", 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1940000000000003e-06 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85008 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662187+0000", 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9995422, 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023390379999999999, 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85008 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.880 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662187+0000", 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662187+0000", 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662186+0000", 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662189+0000", 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2210000000000002e-06 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662196+0000", 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6640000000000001e-06 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662201+0000", 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3499999999999996e-06 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84410 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633350+0000", 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028378865, 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031230985999999999, 2024-09-13T09:26:46.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84410 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633350+0000", 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633350+0000", 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633350+0000", 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633352+0000", 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9539999999999998e-06 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633357+0000", 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5820000000000001e-06 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633361+0000", 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0149999999999997e-06 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84542 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644919+0000", 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016809935000000002, 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019711814000000001, 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84542 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644919+0000", 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644919+0000", 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644919+0000", 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644921+0000", 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3080000000000002e-06 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644926+0000", 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.604e-06 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644931+0000", 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7790000000000003e-06 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84882 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654307+0000", 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0074212870000000004, 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010376254999999999, 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84882 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654307+0000", 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654307+0000", 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654307+0000", 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654310+0000", 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654315+0000", 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4500000000000003e-06 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654320+0000", 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4129999999999999e-06 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.893 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84914 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655899+0000", 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.005829603, 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088325079999999993, 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84914 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655899+0000", 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655899+0000", 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655899+0000", 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655901+0000", 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655906+0000", 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3809999999999998e-06 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655911+0000", 2024-09-13T09:26:46.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6600000000000003e-06 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85010 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662206+0000", 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9995227, 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002578978, 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85010 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662206+0000", 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662206+0000", 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662206+0000", 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662208+0000", 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3860000000000001e-06 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662213+0000", 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662218+0000", 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4560000000000002e-06 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84411 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633365+0000", 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028363431000000001, 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031474993999999999, 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84411 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633365+0000", 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633365+0000", 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633365+0000", 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633367+0000", 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633373+0000", 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633378+0000", 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6229999999999996e-06 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84543 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.904 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644936+0000", 2024-09-13T09:26:46.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016793058, 2024-09-13T09:26:46.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019957259000000001, 2024-09-13T09:26:46.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84543 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644936+0000", 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644936+0000", 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644935+0000", 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644938+0000", 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5000000000000002e-06 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644943+0000", 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3730000000000001e-06 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644947+0000", 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0289999999999999e-06 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.907 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84883 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654561+0000", 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0071680650000000004, 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010380617999999999, 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84883 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654561+0000", 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654561+0000", 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654560+0000", 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654565+0000", 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6810000000000003e-06 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654575+0000", 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.815e-06 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654597+0000", 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1705999999999999e-05 2024-09-13T09:26:46.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84915 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655915+0000", 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0058137019999999996, 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090787239999999998, 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84915 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655915+0000", 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655915+0000", 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655915+0000", 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655917+0000", 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3970000000000001e-06 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655923+0000", 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8980000000000001e-06 2024-09-13T09:26:46.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655928+0000", 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85012 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662222+0000", 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.999507, 2024-09-13T09:26:46.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002821601, 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85012 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662222+0000", 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662222+0000", 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662222+0000", 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662224+0000", 2024-09-13T09:26:46.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4049999999999998e-06 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662229+0000", 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4430000000000002e-06 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662234+0000", 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8049999999999997e-06 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84412 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633381+0000", 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028347496999999999, 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031723125999999997, 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84412 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633381+0000", 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633381+0000", 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633381+0000", 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633383+0000", 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260000000000001e-06 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633388+0000", 2024-09-13T09:26:46.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0930000000000002e-06 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633392+0000", 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2479999999999998e-06 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84544 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644951+0000", 2024-09-13T09:26:46.921 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016777948000000001, 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020202306, 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84544 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644951+0000", 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644951+0000", 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644950+0000", 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644953+0000", 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5459999999999998e-06 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644959+0000", 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9519999999999999e-06 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644964+0000", 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6369999999999998e-06 2024-09-13T09:26:46.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84884 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654604+0000", 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0071249479999999999, 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010605445, 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84884 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654604+0000", 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654604+0000", 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654603+0000", 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654613+0000", 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7999999999999993e-06 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654619+0000", 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3849999999999999e-06 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654625+0000", 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1030000000000001e-06 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84916 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.928 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655933+0000", 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0057961469999999998, 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093477130000000005, 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84916 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655933+0000", 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655933+0000", 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655932+0000", 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655935+0000", 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6240000000000002e-06 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655940+0000", 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4559999999999999e-06 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655945+0000", 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85014 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662238+0000", 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9994903, 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030909050000000001, 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85014 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662238+0000", 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662238+0000", 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662238+0000", 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662240+0000", 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662246+0000", 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5799999999999999e-06 2024-09-13T09:26:46.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662250+0000", 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84413 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633396+0000", 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028332331999999998, 2024-09-13T09:26:46.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032070963000000001, 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84413 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633396+0000", 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633396+0000", 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633396+0000", 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633398+0000", 2024-09-13T09:26:46.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1919999999999999e-06 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633404+0000", 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4879999999999998e-06 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633408+0000", 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3379999999999998e-06 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84545 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.644968+0000", 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.016761150999999998, 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020549218000000001, 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84545 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644968+0000", 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644968+0000", 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644967+0000", 2024-09-13T09:26:46.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.644976+0000", 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1160000000000005e-06 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.646755+0000", 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001778399 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.646765+0000", 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0166000000000001e-05 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84885 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654629+0000", 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0070995679999999997, 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010941157, 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84885 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654629+0000", 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654629+0000", 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654629+0000", 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654632+0000", 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654637+0000", 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654642+0000", 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84917 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.945 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655949+0000", 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0057799720000000004, 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096724189999999998, 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84917 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655949+0000", 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655949+0000", 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655949+0000", 2024-09-13T09:26:46.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655951+0000", 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3089999999999998e-06 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655956+0000", 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.305e-06 2024-09-13T09:26:46.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655961+0000", 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3649999999999997e-06 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85016 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.949 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662255+0000", 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.999474, 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034198240000000001, 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85016 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662255+0000", 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662255+0000", 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662254+0000", 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662257+0000", 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2730000000000001e-06 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662262+0000", 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4779999999999999e-06 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662267+0000", 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3789999999999999e-06 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84414 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633412+0000", 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028316882000000002, 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032312507999999997, 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84414 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633412+0000", 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633412+0000", 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633412+0000", 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633418+0000", 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9430000000000003e-06 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633424+0000", 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9549999999999997e-06 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633429+0000", 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7110000000000003e-06 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84546 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647139+0000", 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014590208, 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018639319000000001, 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84546 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647139+0000", 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647139+0000", 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647138+0000", 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647143+0000", 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0350000000000001e-06 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647151+0000", 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647158+0000", 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4479999999999999e-06 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84886 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654646+0000", 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0070826459999999997, 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011181371000000001, 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84886 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654646+0000", 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654646+0000", 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654646+0000", 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654648+0000", 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7369999999999998e-06 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654654+0000", 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4809999999999997e-06 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654658+0000", 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84918 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655965+0000", 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0057634790000000002, 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099144420000000007, 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84918 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655965+0000", 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655965+0000", 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655964+0000", 2024-09-13T09:26:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655967+0000", 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9890000000000001e-06 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655973+0000", 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655978+0000", 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8130000000000003e-06 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85018 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662271+0000", 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9994578, 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036607100000000002, 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85018 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662271+0000", 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662271+0000", 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662270+0000", 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662273+0000", 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6699999999999998e-06 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662278+0000", 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662283+0000", 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4050000000000002e-06 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84415 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633434+0000", 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028295275000000002, 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032553772000000002, 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84415 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633434+0000", 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633434+0000", 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633433+0000", 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633435+0000", 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633441+0000", 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633445+0000", 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:26:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84547 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647163+0000", 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014565754, 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.018875559, 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84547 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647163+0000", 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647163+0000", 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647163+0000", 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647166+0000", 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0819999999999999e-06 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647172+0000", 2024-09-13T09:26:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9190000000000004e-06 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647177+0000", 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1200000000000001e-06 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84887 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654676+0000", 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0070527740000000004, 2024-09-13T09:26:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011412327999999999, 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84887 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654676+0000", 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654676+0000", 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654676+0000", 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654678+0000", 2024-09-13T09:26:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654684+0000", 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654689+0000", 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7280000000000003e-06 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84919 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655982+0000", 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00574634, 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010161549000000001, 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84919 2024-09-13T09:26:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655982+0000", 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655982+0000", 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655982+0000", 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655985+0000", 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655990+0000", 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4709999999999998e-06 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655995+0000", 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.826e-06 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85019 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662311+0000", 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9994183, 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038842299999999998, 2024-09-13T09:26:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85019 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662311+0000", 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662311+0000", 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662310+0000", 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662313+0000", 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8600000000000001e-06 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662319+0000", 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0820000000000004e-06 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662324+0000", 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0830000000000003e-06 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84416 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633449+0000", 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028279578999999999, 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03279783, 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84416 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633449+0000", 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633449+0000", 2024-09-13T09:26:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633449+0000", 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633451+0000", 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1940000000000001e-06 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633456+0000", 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633461+0000", 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5299999999999998e-06 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84548 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647181+0000", 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014547895, 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019118155000000001, 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84548 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647181+0000", 2024-09-13T09:26:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647181+0000", 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647181+0000", 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647183+0000", 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3439999999999999e-06 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647188+0000", 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647193+0000", 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-13T09:26:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84888 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654693+0000", 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0070352949999999996, 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011657829999999999, 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84888 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654693+0000", 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654693+0000", 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654693+0000", 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654696+0000", 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654701+0000", 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5609999999999998e-06 2024-09-13T09:26:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654706+0000", 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4440000000000001e-06 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84920 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655999+0000", 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.005729652, 2024-09-13T09:26:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010402369999999999, 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84920 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655999+0000", 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655999+0000", 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655999+0000", 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656001+0000", 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4250000000000001e-06 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656006+0000", 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3120000000000001e-06 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656011+0000", 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85020 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662329+0000", 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9994001, 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041285389999999996, 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85020 2024-09-13T09:26:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662329+0000", 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662329+0000", 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662328+0000", 2024-09-13T09:26:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662331+0000", 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.491e-06 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662336+0000", 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4160000000000003e-06 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662340+0000", 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2479999999999998e-06 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84417 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633465+0000", 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028264022999999999, 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033042674000000001, 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84417 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633465+0000", 2024-09-13T09:26:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633465+0000", 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633464+0000", 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633467+0000", 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1919999999999999e-06 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633472+0000", 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2279999999999998e-06 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633476+0000", 2024-09-13T09:26:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9720000000000003e-06 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84549 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647196+0000", 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014532362, 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019363689, 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84549 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647196+0000", 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647196+0000", 2024-09-13T09:26:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647196+0000", 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647198+0000", 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3879999999999998e-06 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647204+0000", 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647209+0000", 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7439999999999998e-06 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84889 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.654710+0000", 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0070190469999999996, 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011900043000000001, 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84889 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654710+0000", 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654710+0000", 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654709+0000", 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654712+0000", 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654717+0000", 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5280000000000003e-06 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.654722+0000", 2024-09-13T09:26:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.566e-06 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84921 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656015+0000", 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0057138190000000002, 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01064671, 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84921 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656015+0000", 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656015+0000", 2024-09-13T09:26:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656015+0000", 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656027+0000", 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2449000000000001e-05 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656034+0000", 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4119999999999998e-06 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656038+0000", 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85021 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662344+0000", 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9993844, 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043669410000000001, 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85021 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662344+0000", 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662344+0000", 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662344+0000", 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662346+0000", 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662351+0000", 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1819999999999997e-06 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662355+0000", 2024-09-13T09:26:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1529999999999999e-06 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84418 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633480+0000", 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028248615000000001, 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033280500999999997, 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84418 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633480+0000", 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633480+0000", 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633480+0000", 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633482+0000", 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633487+0000", 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3079999999999998e-06 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633492+0000", 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5140000000000003e-06 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84550 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647213+0000", 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014515853, 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019601786999999999, 2024-09-13T09:26:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84550 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647213+0000", 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647213+0000", 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647213+0000", 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647215+0000", 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.475e-06 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647220+0000", 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647225+0000", 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6140000000000002e-06 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84890 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655390+0000", 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.006338303, 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011477062999999999, 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84890 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655390+0000", 2024-09-13T09:26:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655390+0000", 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655389+0000", 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655395+0000", 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4489999999999998e-06 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655405+0000", 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0457e-05 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655413+0000", 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2690000000000002e-06 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84922 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656044+0000", 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0056848710000000002, 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010875999000000001, 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84922 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656044+0000", 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656044+0000", 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656044+0000", 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656046+0000", 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656052+0000", 2024-09-13T09:26:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7610000000000004e-06 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656058+0000", 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.665e-06 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85022 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662359+0000", 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9993696, 2024-09-13T09:26:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046104450000000003, 2024-09-13T09:26:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85022 2024-09-13T09:26:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662359+0000", 2024-09-13T09:26:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662359+0000", 2024-09-13T09:26:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662359+0000", 2024-09-13T09:26:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662361+0000", 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4109999999999999e-06 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662367+0000", 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7599999999999999e-06 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662372+0000", 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8149999999999996e-06 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84387 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 781312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.632906+0000", 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028822803000000001, 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034124336999999998, 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84387 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632906+0000", 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632906+0000", 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632906+0000", 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632908+0000", 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632924+0000", 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5990000000000001e-05 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632929+0000", 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0860000000000001e-06 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664941+0000", 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032012497000000001 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.665008+0000", 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6777000000000001e-05 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84419 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633496+0000", 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028232739999999999, 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033758614999999999, 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84419 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633496+0000", 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633496+0000", 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633496+0000", 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633498+0000", 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633503+0000", 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9989999999999999e-06 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633508+0000", 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6859999999999997e-06 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84551 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647229+0000", 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014499779000000001, 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020079840000000002, 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84551 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647229+0000", 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647229+0000", 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647229+0000", 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647231+0000", 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647237+0000", 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647241+0000", 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3660000000000002e-06 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84891 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655420+0000", 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0063089790000000001, 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011956714, 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84891 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655420+0000", 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655420+0000", 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655419+0000", 2024-09-13T09:26:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655436+0000", 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6424000000000001e-05 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655443+0000", 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4309999999999997e-06 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655449+0000", 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8540000000000002e-06 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84923 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656062+0000", 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0056670480000000001, 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011367119, 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84923 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656062+0000", 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656062+0000", 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656061+0000", 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656064+0000", 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3290000000000001e-06 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656069+0000", 2024-09-13T09:26:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.288e-06 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656074+0000", 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85023 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662376+0000", 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9993525, 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051015059999999996, 2024-09-13T09:26:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85023 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662376+0000", 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662376+0000", 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662376+0000", 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662379+0000", 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3989999999999998e-06 2024-09-13T09:26:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662418+0000", 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9007000000000001e-05 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662422+0000", 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7280000000000003e-06 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84388 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 688128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.632934+0000", 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028794717000000001, 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034612154999999999, 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84388 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632934+0000", 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632934+0000", 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632934+0000", 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632937+0000", 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9299999999999999e-06 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632942+0000", 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6250000000000004e-06 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.632947+0000", 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4020000000000004e-06 2024-09-13T09:26:47.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.667418+0000", 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034471361999999998 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.667487+0000", 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8654999999999999e-05 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84420 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633512+0000", 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028217222, 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034094474, 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84420 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633512+0000", 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633512+0000", 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633511+0000", 2024-09-13T09:26:47.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633513+0000", 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.181e-06 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633518+0000", 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:26:47.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633523+0000", 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0500000000000002e-06 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84552 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647245+0000", 2024-09-13T09:26:47.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014484076, 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020412844999999999, 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84552 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647245+0000", 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647245+0000", 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647244+0000", 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647247+0000", 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1919999999999999e-06 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647252+0000", 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0880000000000002e-06 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647256+0000", 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5460000000000002e-06 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84892 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655454+0000", 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0062743030000000002, 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012252581, 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84892 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655454+0000", 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655454+0000", 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655454+0000", 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655457+0000", 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.818e-06 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655462+0000", 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4280000000000004e-06 2024-09-13T09:26:47.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655474+0000", 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1405e-05 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84924 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656079+0000", 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0056498260000000002, 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01168016, 2024-09-13T09:26:47.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84924 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656079+0000", 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656079+0000", 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656079+0000", 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656081+0000", 2024-09-13T09:26:47.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.221e-06 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656086+0000", 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4670000000000003e-06 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656091+0000", 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5120000000000002e-06 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85024 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662427+0000", 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9993019, 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053812290000000004, 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85024 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662427+0000", 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662427+0000", 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662427+0000", 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662430+0000", 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662436+0000", 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4930000000000003e-06 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662441+0000", 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8350000000000003e-06 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84422 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633526+0000", 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028202439999999999, 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034338199, 2024-09-13T09:26:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84422 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633526+0000", 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633526+0000", 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633526+0000", 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633528+0000", 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2000000000000001e-06 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633534+0000", 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5389999999999998e-06 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633538+0000", 2024-09-13T09:26:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4469999999999999e-06 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84553 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647260+0000", 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014468606, 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02065345, 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84553 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647260+0000", 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647260+0000", 2024-09-13T09:26:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647260+0000", 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647262+0000", 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3319999999999999e-06 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647268+0000", 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4960000000000004e-06 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647272+0000", 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4630000000000003e-06 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84893 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655478+0000", 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0062503100000000002, 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012483519, 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84893 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655478+0000", 2024-09-13T09:26:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655478+0000", 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655478+0000", 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655481+0000", 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.977e-06 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655487+0000", 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7860000000000002e-06 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655492+0000", 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84925 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656095+0000", 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0056339729999999996, 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011921556999999999, 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84925 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656095+0000", 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656095+0000", 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656094+0000", 2024-09-13T09:26:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656097+0000", 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656102+0000", 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3290000000000001e-06 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656107+0000", 2024-09-13T09:26:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.515e-06 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85025 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662445+0000", 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9992833, 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005620852, 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85025 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662445+0000", 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662445+0000", 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662445+0000", 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662459+0000", 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3536999999999999e-05 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662465+0000", 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.195e-06 2024-09-13T09:26:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662470+0000", 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7199999999999997e-06 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84423 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633542+0000", 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028186705999999999, 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034580885999999998, 2024-09-13T09:26:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84423 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633542+0000", 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633542+0000", 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633542+0000", 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633544+0000", 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.204e-06 2024-09-13T09:26:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633549+0000", 2024-09-13T09:26:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633553+0000", 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1239999999999998e-06 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84554 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647276+0000", 2024-09-13T09:26:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014452678, 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.020896106000000001, 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84554 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647276+0000", 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647276+0000", 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647276+0000", 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647278+0000", 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1959999999999998e-06 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647283+0000", 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9819999999999999e-06 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647287+0000", 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0500000000000002e-06 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84894 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655498+0000", 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0062308169999999996, 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012726636, 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84894 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655498+0000", 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655498+0000", 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655497+0000", 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655503+0000", 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0109999999999997e-06 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655512+0000", 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5019999999999992e-06 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655517+0000", 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84926 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656111+0000", 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.005618245, 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012162826999999999, 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84926 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656111+0000", 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656111+0000", 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656110+0000", 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656112+0000", 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656118+0000", 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3199999999999999e-06 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656122+0000", 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4669999999999998e-06 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85026 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662474+0000", 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9992547, 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058532360000000004, 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85026 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662474+0000", 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662474+0000", 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662474+0000", 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662476+0000", 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.649e-06 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662482+0000", 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4530000000000001e-06 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662486+0000", 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.403e-06 2024-09-13T09:26:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84425 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633557+0000", 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028171695, 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034824230999999997, 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84425 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633557+0000", 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633557+0000", 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633557+0000", 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633559+0000", 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2060000000000001e-06 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633564+0000", 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2809999999999999e-06 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633569+0000", 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3610000000000003e-06 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84555 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647291+0000", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014437293, 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021141558000000001, 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84555 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647291+0000", 2024-09-13T09:26:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647291+0000", 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647291+0000", 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647293+0000", 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647299+0000", 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647304+0000", 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9540000000000003e-06 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84895 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655555+0000", 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0061733639999999998, 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012927855, 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84895 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655555+0000", 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655555+0000", 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655555+0000", 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655558+0000", 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4879999999999999e-06 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655565+0000", 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655571+0000", 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5860000000000004e-06 2024-09-13T09:26:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84927 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656127+0000", 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0056021839999999996, 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012461837, 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84927 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656127+0000", 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656127+0000", 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656126+0000", 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656129+0000", 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5749999999999999e-06 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656134+0000", 2024-09-13T09:26:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3619999999999997e-06 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656139+0000", 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85027 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662490+0000", 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9992385, 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061551610000000001, 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85027 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662490+0000", 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662490+0000", 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662490+0000", 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662492+0000", 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4269999999999998e-06 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662497+0000", 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662502+0000", 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8949999999999997e-06 2024-09-13T09:26:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84427 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633573+0000", 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028156111000000001, 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035127129, 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84427 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633573+0000", 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633573+0000", 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633572+0000", 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633575+0000", 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3489999999999999e-06 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633580+0000", 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633584+0000", 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84556 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647308+0000", 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014421069, 2024-09-13T09:26:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021443964999999999, 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84556 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647308+0000", 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647308+0000", 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647307+0000", 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647310+0000", 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3130000000000001e-06 2024-09-13T09:26:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647315+0000", 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1220000000000003e-06 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647319+0000", 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4209999999999997e-06 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84896 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655576+0000", 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0061527939999999996, 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013238002, 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84896 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655576+0000", 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655576+0000", 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655576+0000", 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655578+0000", 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2989999999999999e-06 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655583+0000", 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.322e-06 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655588+0000", 2024-09-13T09:26:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.498e-06 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84928 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656143+0000", 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0055858519999999997, 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012723895000000001, 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84928 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656143+0000", 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656143+0000", 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656143+0000", 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656145+0000", 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4310000000000001e-06 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656150+0000", 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.242e-06 2024-09-13T09:26:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656155+0000", 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5310000000000003e-06 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85028 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662506+0000", 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9992228, 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064153420000000001, 2024-09-13T09:26:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85028 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662506+0000", 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662506+0000", 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662506+0000", 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662508+0000", 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2390000000000001e-06 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662513+0000", 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1930000000000001e-06 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662517+0000", 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2100000000000003e-06 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84429 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633588+0000", 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028140404000000001, 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035389957, 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84429 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633588+0000", 2024-09-13T09:26:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633588+0000", 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633588+0000", 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633590+0000", 2024-09-13T09:26:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633596+0000", 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3329999999999996e-06 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633600+0000", 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2780000000000004e-06 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84557 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647323+0000", 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014405738, 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021704358, 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84557 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647323+0000", 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647323+0000", 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647323+0000", 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647331+0000", 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7369999999999995e-06 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647344+0000", 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2394e-05 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647349+0000", 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.147e-06 2024-09-13T09:26:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84897 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655591+0000", 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0061373260000000002, 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013488005000000001, 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84897 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655591+0000", 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655591+0000", 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655591+0000", 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655593+0000", 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260000000000001e-06 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655599+0000", 2024-09-13T09:26:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7230000000000001e-06 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655603+0000", 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.284e-06 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84929 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656159+0000", 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0055697960000000001, 2024-09-13T09:26:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012970216999999999, 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84929 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656159+0000", 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656159+0000", 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656159+0000", 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656161+0000", 2024-09-13T09:26:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2970000000000002e-06 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656166+0000", 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5999999999999997e-06 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656171+0000", 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5519999999999998e-06 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85029 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662522+0000", 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.999207, 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066594200000000001, 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85029 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662522+0000", 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662522+0000", 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662521+0000", 2024-09-13T09:26:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662524+0000", 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.43e-06 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662529+0000", 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3210000000000004e-06 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662533+0000", 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.172e-06 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84431 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633604+0000", 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028125094, 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035637851999999998, 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84431 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633604+0000", 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633604+0000", 2024-09-13T09:26:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633603+0000", 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633610+0000", 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7170000000000002e-06 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633616+0000", 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633620+0000", 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6160000000000004e-06 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84558 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647403+0000", 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014325981999999999, 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.021888508000000001, 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84558 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647403+0000", 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647403+0000", 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647402+0000", 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647406+0000", 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2239999999999997e-06 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647414+0000", 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647419+0000", 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5960000000000003e-06 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84898 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655608+0000", 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0061208010000000004, 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013878011000000001, 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84898 2024-09-13T09:26:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655608+0000", 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655608+0000", 2024-09-13T09:26:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655608+0000", 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655610+0000", 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655617+0000", 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3430000000000001e-06 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655622+0000", 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6409999999999999e-06 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84930 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656175+0000", 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0055537570000000003, 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013360278, 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84930 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656175+0000", 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656175+0000", 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656175+0000", 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656177+0000", 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2340000000000001e-06 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656183+0000", 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656187+0000", 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.352e-06 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85030 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662537+0000", 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9991913, 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070500470000000003, 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85030 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662537+0000", 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662537+0000", 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662537+0000", 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662540+0000", 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662545+0000", 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3770000000000004e-06 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662550+0000", 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.702e-06 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84433 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633625+0000", 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028104275000000001, 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036017262000000001, 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84433 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633625+0000", 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633625+0000", 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633624+0000", 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633626+0000", 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2079999999999999e-06 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633632+0000", 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633636+0000", 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0960000000000003e-06 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84559 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647423+0000", 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.01430534, 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022270294999999999, 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84559 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647423+0000", 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647423+0000", 2024-09-13T09:26:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647423+0000", 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647426+0000", 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.413e-06 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647431+0000", 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8359999999999997e-06 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647436+0000", 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84899 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655636+0000", 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0060927050000000003, 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014105991999999999, 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84899 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655636+0000", 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655636+0000", 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655636+0000", 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655639+0000", 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0230000000000001e-06 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655645+0000", 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0000000000000002e-06 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655649+0000", 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84931 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656191+0000", 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0055375980000000003, 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013602804, 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84931 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.197 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656191+0000", 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656191+0000", 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656191+0000", 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656193+0000", 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.356e-06 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656199+0000", 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656204+0000", 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8509999999999998e-06 2024-09-13T09:26:47.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85031 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662554+0000", 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9991751, 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072902720000000004, 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85031 2024-09-13T09:26:47.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662554+0000", 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662554+0000", 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662553+0000", 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662556+0000", 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2359999999999999e-06 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662563+0000", 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0990000000000001e-06 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662567+0000", 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84435 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.633640+0000", 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.028089053999999999, 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036259975999999999, 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84435 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633640+0000", 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633640+0000", 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633639+0000", 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633642+0000", 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2139999999999999e-06 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633647+0000", 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3990000000000003e-06 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.633651+0000", 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1239999999999998e-06 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84560 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647441+0000", 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014288210000000001, 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022507833000000001, 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84560 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.207 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647441+0000", 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647441+0000", 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647440+0000", 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647443+0000", 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647449+0000", 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647453+0000", 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4490000000000001e-06 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84900 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655654+0000", 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0060748119999999997, 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014348734, 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84900 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655654+0000", 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655654+0000", 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655653+0000", 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655656+0000", 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655661+0000", 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.361e-06 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655666+0000", 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.213 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84932 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656208+0000", 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0055208439999999996, 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.013844543000000001, 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84932 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656208+0000", 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656208+0000", 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656208+0000", 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656210+0000", 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656216+0000", 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5419999999999996e-06 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656220+0000", 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4920000000000004e-06 2024-09-13T09:26:47.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85032 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662571+0000", 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9991574, 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075335589999999996, 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85032 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662571+0000", 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662571+0000", 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662571+0000", 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662573+0000", 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2809999999999998e-06 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662579+0000", 2024-09-13T09:26:47.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.755e-06 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662584+0000", 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8400000000000002e-06 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84561 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647458+0000", 2024-09-13T09:26:47.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014271123, 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022698676000000001, 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84561 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647458+0000", 2024-09-13T09:26:47.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647458+0000", 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647457+0000", 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647460+0000", 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.43e-06 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647465+0000", 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6590000000000004e-06 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647470+0000", 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5580000000000002e-06 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84901 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655670+0000", 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0060587519999999997, 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014536668000000001, 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84901 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655670+0000", 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655670+0000", 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655670+0000", 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655672+0000", 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655677+0000", 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3469999999999998e-06 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655682+0000", 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84934 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.227 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656224+0000", 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0055046959999999999, 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014034181, 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84934 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656224+0000", 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656224+0000", 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656224+0000", 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656236+0000", 2024-09-13T09:26:47.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2228999999999999e-05 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656242+0000", 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656249+0000", 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6540000000000002e-06 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85033 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.662789+0000", 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.99894, 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075210019999999997, 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85033 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662789+0000", 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662789+0000", 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662788+0000", 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662792+0000", 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1860000000000002e-06 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662800+0000", 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3540000000000002e-06 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.662806+0000", 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9540000000000001e-06 2024-09-13T09:26:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84562 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.647474+0000", 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0142545, 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022890938999999999, 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84562 2024-09-13T09:26:47.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647474+0000", 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647474+0000", 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647474+0000", 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647476+0000", 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.57e-06 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647482+0000", 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5010000000000004e-06 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.647487+0000", 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6619999999999996e-06 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84902 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.655686+0000", 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0060427370000000003, 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014749307, 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84902 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655686+0000", 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655686+0000", 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655686+0000", 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655688+0000", 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2479999999999999e-06 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655693+0000", 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1630000000000004e-06 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.655699+0000", 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7339999999999996e-06 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84936 3.15 3.ee762f75 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.656254+0000", 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0054748030000000003, 2024-09-13T09:26:47.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014234261, 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84936 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656254+0000", 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656254+0000", 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656254+0000", 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656256+0000", 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7240000000000001e-06 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656262+0000", 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8629999999999996e-06 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.656267+0000", 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6249999999999998e-06 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85034 3.e 3.386f942e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.663277+0000", 2024-09-13T09:26:47.243 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9984522, 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0072677590000000004, 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85034 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.663277+0000", 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.663277+0000", 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.663276+0000", 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.663281+0000", 2024-09-13T09:26:47.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4900000000000002e-06 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.663289+0000", 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4130000000000006e-06 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.663295+0000", 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3099999999999997e-06 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.246 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:26:47.247 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 160 2024-09-13T09:26:47.247 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:47.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:2257 trunc from 0xe64c65c to 0x7e7b69a 2024-09-13T09:26:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:2263 writesame 0xd61d1b7 thru 0xd624bcf data_size 0x45(0x7a19 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2264 write 0x7864a80 thru 0x786d444 (0x89c5 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2266 write 0x75705c8 thru 0x757feaa (0xf8e3 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2267 read 0xd1bb739 thru 0xd1c4eda (0x97a2 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2268 write 0x4d1190d thru 0x4d14bfc (0x32f0 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2269 read 0x59fafb7 thru 0x59fca2f (0x1a79 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2271 read 0xa3092b6 thru 0xa3125ad (0x92f8 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2272 trunc from 0xd624bd0 to 0xe1a2f8 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2275 read 0x4881c3 thru 0x490446 (0x8284 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2277 write 0x2c3e77e thru 0x2c4b40b (0xcc8e bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:2278 writesame 0x4d9d8e2 thru 0x4d9e394 data_size 0x21(0xab3 bytes) 2024-09-13T09:26:47.252 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2280 writesame 0xab5250d thru 0xab53179 data_size 0x1(0xc6d bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2283 read 0x1490471 thru 0x1497154 (0x6ce4 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2285 punch from 0x129cbf4 to 0x12a41ef, (0x75fb bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2287 writesame 0x483286a thru 0x4838aa8 data_size 0x1(0x623f bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2288 punch from 0x778a0a9 to 0x7790bbf, (0x6b16 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2289 trunc from 0xab5317a to 0xe9a0470 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2290 punch from 0x5224069 to 0x5226416, (0x23ad bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2291 write 0x837e46f thru 0x8386d8c (0x891e bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2293 punch from 0xc790b00 to 0xc79b152, (0xa652 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2294 writesame 0xc291731 thru 0xc294596 data_size 0x1(0x2e66 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2295 writesame 0xbf27825 thru 0xbf29de8 data_size 0x2(0x25c4 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2296 write 0xb7e0dd2 thru 0xb7e27c8 (0x19f7 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2297 punch from 0xd8016c8 to 0xd807209, (0x5b41 bytes) 2024-09-13T09:26:47.253 INFO:teuthology.orchestra.run.smithi063.stdout:2299 write 0x9a18c68 thru 0x9a1bcb4 (0x304d bytes) 2024-09-13T09:26:47.254 INFO:teuthology.orchestra.run.smithi063.stdout:2300 trunc from 0xe9a0470 to 0xad256d9 2024-09-13T09:26:47.254 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:47.239+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:47.262 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.277 INFO:teuthology.orchestra.run.smithi063.stdout:2302 writesame 0xdb6d87b thru 0xdb799dc data_size 0x25(0xc162 bytes) 2024-09-13T09:26:47.277 INFO:teuthology.orchestra.run.smithi063.stdout:2303 writesame 0xbb19286 thru 0xbb21c3e data_size 0x1(0x89b9 bytes) 2024-09-13T09:26:47.318 INFO:teuthology.orchestra.run.smithi063.stdout:2304 trunc from 0xdb799dd to 0x85d748f 2024-09-13T09:26:47.343 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.349 INFO:teuthology.orchestra.run.smithi063.stdout:2306 trunc from 0x85d748f to 0x50e18c0 2024-09-13T09:26:47.350 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:47.343+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:47.365 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:47.365 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:47.365 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:47.365 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:47.365 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:47.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:47.383 INFO:teuthology.orchestra.run.smithi063.stdout:2307 write 0x64528c5 thru 0x6459bab (0x72e7 bytes) 2024-09-13T09:26:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:2309 punch from 0x84d4ce to 0x8573dc, (0x9f0e bytes) 2024-09-13T09:26:47.418 INFO:teuthology.orchestra.run.smithi063.stdout:2310 write 0x67cecfc thru 0x67d5c90 (0x6f95 bytes) 2024-09-13T09:26:47.445 INFO:teuthology.orchestra.run.smithi063.stdout:2311 write 0x675eaca thru 0x676795e (0x8e95 bytes) 2024-09-13T09:26:47.449 INFO:teuthology.orchestra.run.smithi063.stdout:2312 read 0x62dc7c6 thru 0x62e3b6b (0x73a6 bytes) 2024-09-13T09:26:47.450 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:47.443+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:2314 read 0x4ceb1f7 thru 0x4cfa77e (0xf588 bytes) 2024-09-13T09:26:47.451 INFO:teuthology.orchestra.run.smithi063.stdout:2315 writesame 0x2dc0242 thru 0x2dca846 data_size 0x1f(0xa605 bytes) 2024-09-13T09:26:47.488 INFO:teuthology.orchestra.run.smithi063.stdout:2316 writesame 0x771a9fa thru 0x7727d69 data_size 0x44(0xd370 bytes) 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 64.531509659999998, 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.418289916, 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:26:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:26:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.417891901, 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:26:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74278502499999999, 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:26:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74277009299999996, 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:26:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:26:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:26:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74273698700000002, 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74272069399999996, 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:26:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74270457899999998, 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:26:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:26:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:26:47.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74267353800000002, 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.74264191499999999, 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:26:47.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.544 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.731748603, 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:26:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.73172826599999996, 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:26:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:26:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72969385399999998, 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:26:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:26:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72967679799999996, 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:26:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72966063000000003, 2024-09-13T09:26:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:26:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72944696799999997, 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:26:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72942765700000001, 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.729411264, 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:26:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72939560199999998, 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.72936924800000003, 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:26:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:47.589 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:47.590 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:47.591 INFO:teuthology.orchestra.run.smithi063.stdout:2318 trunc from 0x7727d6a to 0x9d4069e 2024-09-13T09:26:47.591 INFO:teuthology.orchestra.run.smithi063.stdout:2319 write 0xa0eaac4 thru 0xa0f00ad (0x55ea bytes) 2024-09-13T09:26:47.591 INFO:teuthology.orchestra.run.smithi063.stdout:2320 write 0xb43305b thru 0xb436680 (0x3626 bytes) 2024-09-13T09:26:47.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:47.593 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:47.583+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:47.596 INFO:teuthology.orchestra.run.smithi063.stdout:2321 writesame 0x58fa7a2 thru 0x59085d3 data_size 0x22(0xde32 bytes) 2024-09-13T09:26:47.622 INFO:teuthology.orchestra.run.smithi063.stdout:2322 write 0x8b8a452 thru 0x8b8ec9c (0x484b bytes) 2024-09-13T09:26:47.648 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.651 INFO:teuthology.orchestra.run.smithi063.stdout:2325 trunc from 0xb436681 to 0xc804564 2024-09-13T09:26:47.660 INFO:teuthology.orchestra.run.smithi063.stdout:2326 read 0xc5dd69b thru 0xc5e4d44 (0x76aa bytes) 2024-09-13T09:26:47.660 INFO:teuthology.orchestra.run.smithi063.stdout:2327 writesame 0xc4997bb thru 0xc49c337 data_size 0x3(0x2b7d bytes) 2024-09-13T09:26:47.692 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:47.688+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:47.692 INFO:teuthology.orchestra.run.smithi063.stdout:2328 write 0xd9f7a3c thru 0xda068e9 (0xeeae bytes) 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout:2331 write 0x992488 thru 0x995006 (0x2b7f bytes) 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:47.721 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:47.727 INFO:teuthology.orchestra.run.smithi063.stdout:2332 read 0x5200c7b thru 0x5202912 (0x1c98 bytes) 2024-09-13T09:26:47.727 INFO:teuthology.orchestra.run.smithi063.stdout:2334 read 0x9d7f9d5 thru 0x9d829f8 (0x3024 bytes) 2024-09-13T09:26:47.734 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:47.737 INFO:teuthology.orchestra.run.smithi063.stdout:2335 writesame 0xe723a1a thru 0xe723dfc data_size 0x1(0x3e3 bytes) 2024-09-13T09:26:47.742 INFO:teuthology.orchestra.run.smithi063.stdout:2336 trunc from 0xe723dfd to 0xbedfaa7 2024-09-13T09:26:47.742 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:47.767 INFO:teuthology.orchestra.run.smithi063.stdout:2339 write 0xafaae9d thru 0xafb9ba6 (0xed0a bytes) 2024-09-13T09:26:47.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:47.790 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:47.793 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:47.792+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:47.799 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.817 INFO:teuthology.orchestra.run.smithi063.stdout:2341 trunc from 0xbedfaa7 to 0x124b133 2024-09-13T09:26:47.818 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:47.849 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:47.861 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:47.861 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:47.866 INFO:teuthology.orchestra.run.smithi063.stdout:2342 read 0x1048e29 thru 0x104be7e (0x3056 bytes) 2024-09-13T09:26:47.866 INFO:teuthology.orchestra.run.smithi063.stdout:2343 read 0x4c3613 thru 0x4c3ebb (0x8a9 bytes) 2024-09-13T09:26:47.867 INFO:teuthology.orchestra.run.smithi063.stdout:2344 read 0x3ed91e thru 0x3f7826 (0x9f09 bytes) 2024-09-13T09:26:47.868 INFO:teuthology.orchestra.run.smithi063.stdout:2346 punch from 0x163d19 to 0x16ab7b, (0x6e62 bytes) 2024-09-13T09:26:47.868 INFO:teuthology.orchestra.run.smithi063.stdout:2347 punch from 0x4a63f7 to 0x4b124b, (0xae54 bytes) 2024-09-13T09:26:47.872 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:47.872 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:47.886 INFO:teuthology.orchestra.run.smithi063.stdout:2349 write 0x72ebe0b thru 0x72f2d6d (0x6f63 bytes) 2024-09-13T09:26:47.894 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:47.894 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:47.895 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:47.887+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:47.897 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:47.898 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:47.916 INFO:teuthology.orchestra.run.smithi063.stdout:2351 punch from 0x6b50f05 to 0x6b5ef1c, (0xe017 bytes) 2024-09-13T09:26:47.916 INFO:teuthology.orchestra.run.smithi063.stdout:2352 read 0x44a733f thru 0x44ad2ee (0x5fb0 bytes) 2024-09-13T09:26:47.935 INFO:teuthology.orchestra.run.smithi063.stdout:2354 write 0xc3a67f8 thru 0xc3b00c9 (0x98d2 bytes) 2024-09-13T09:26:47.963 INFO:teuthology.orchestra.run.smithi063.stdout:2357 read 0xb9fabae thru 0xb9fb316 (0x769 bytes) 2024-09-13T09:26:47.963 INFO:teuthology.orchestra.run.smithi063.stdout:2361 read 0xc38bc7a thru 0xc39acbe (0xf045 bytes) 2024-09-13T09:26:47.964 INFO:teuthology.orchestra.run.smithi063.stdout:2362 punch from 0x657d7ac to 0x6586d37, (0x958b bytes) 2024-09-13T09:26:47.965 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:47.965 INFO:teuthology.orchestra.run.smithi063.stdout:2367 writesame 0x42447ed thru 0x42536e9 data_size 0x1(0xeefd bytes) 2024-09-13T09:26:47.992 INFO:teuthology.orchestra.run.smithi063.stdout:2369 punch from 0x4f8287 to 0x505ec8, (0xdc41 bytes) 2024-09-13T09:26:47.992 INFO:teuthology.orchestra.run.smithi063.stdout:2372 write 0xe89dd9 thru 0xe8b111 (0x1339 bytes) 2024-09-13T09:26:47.994 INFO:teuthology.orchestra.run.smithi063.stdout:2373 write 0x5a5c727 thru 0x5a64626 (0x7f00 bytes) 2024-09-13T09:26:47.995 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:47.992+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:47.997 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:47.997 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.008 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:48.033 INFO:teuthology.orchestra.run.smithi063.stdout:2374 trunc from 0xc3b00ca to 0x39961cc 2024-09-13T09:26:48.073 INFO:teuthology.orchestra.run.smithi063.stdout:2375 write 0x66f8c21 thru 0x6704e51 (0xc231 bytes) 2024-09-13T09:26:48.095 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:48.087+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:48.100 INFO:teuthology.orchestra.run.smithi063.stdout:2377 read 0x3a60353 thru 0x3a691cd (0x8e7b bytes) 2024-09-13T09:26:48.100 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:48.117 INFO:teuthology.orchestra.run.smithi063.stdout:2379 write 0xa95de16 thru 0xa95e776 (0x961 bytes) 2024-09-13T09:26:48.127 INFO:teuthology.orchestra.run.smithi063.stdout:2380 write 0x3c9e4f2 thru 0x3ca2a8a (0x4599 bytes) 2024-09-13T09:26:48.144 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.144 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.157 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:48.161 INFO:teuthology.orchestra.run.smithi063.stdout:2382 trunc from 0xa95e777 to 0x776a331 2024-09-13T09:26:48.197 INFO:teuthology.orchestra.run.smithi063.stdout:2383 write 0xd29ba43 thru 0xd29e978 (0x2f36 bytes) 2024-09-13T09:26:48.197 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:48.191+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:48.226 INFO:teuthology.orchestra.run.smithi063.stdout:2384 clone 21 order 23 su 131072 sc 7 2024-09-13T09:26:48.270 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.270 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:48.321 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:48.315+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:48.400 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:48.396+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:48.432 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.432 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.441 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:48.501 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:48.500+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:48.552 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.552 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.562 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:48.602 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:48.600+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:48.615 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:26:48.687 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.687 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.697 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:48.703 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:48.700+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:48.745 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone20 from 0xd29e979 (overlap 0xe1a2f8) to 0xb48260 2024-09-13T09:26:48.804 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:48.795+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:48.813 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.814 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:48.905 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:48.900+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:48.943 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:48.943 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:48.946 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T09:26:48.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:49.005 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:49.004+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:49.059 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.059 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.069 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:49.106 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:49.099+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:49.207 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:49.204+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:49.229 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.230 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:49.303 INFO:teuthology.orchestra.run.smithi063.stdout:2385 writesame 0x7dbd140 thru 0x7dc0bc3 data_size 0xa(0x3a84 bytes) 2024-09-13T09:26:49.308 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:49.299+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:49.333 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:49.333 INFO:teuthology.orchestra.run.smithi063.stdout:2388 write 0x6a48a8 thru 0x6ab4dd (0x6c36 bytes) 2024-09-13T09:26:49.361 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.361 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:49.415 INFO:teuthology.orchestra.run.smithi063.stdout:2389 read 0x91cc4b8 thru 0x91d2e7c (0x69c5 bytes) 2024-09-13T09:26:49.416 INFO:teuthology.orchestra.run.smithi063.stdout:2390 read 0xdc3704 thru 0xdc8093 (0x4990 bytes) 2024-09-13T09:26:49.417 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:49.416+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:49.427 INFO:teuthology.orchestra.run.smithi063.stdout:2393 read 0x2521dca thru 0x252494d (0x2b84 bytes) 2024-09-13T09:26:49.439 INFO:teuthology.orchestra.run.smithi063.stdout:2395 punch from 0x94fb6fb to 0x9504e7b, (0x9780 bytes) 2024-09-13T09:26:49.455 INFO:teuthology.orchestra.run.smithi063.stdout:2396 read 0xe43cf1 thru 0xe48ff5 (0x5305 bytes) 2024-09-13T09:26:49.492 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.493 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:49.508 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:49.504+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.597 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.606 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:49.610 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:49.603+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:49.622 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:26:49.687 INFO:teuthology.orchestra.run.smithi063.stdout:2397 read 0x66fe097 thru 0x670d6ca (0xf634 bytes) 2024-09-13T09:26:49.710 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:49.703+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.733 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:49.751 INFO:teuthology.orchestra.run.smithi063.stdout:2399 punch from 0x7b960d8 to 0x7b9618e, (0xb6 bytes) 2024-09-13T09:26:49.761 INFO:teuthology.orchestra.run.smithi063.stdout:2401 punch from 0xb1731bc to 0xb17ce96, (0x9cda bytes) 2024-09-13T09:26:49.762 INFO:teuthology.orchestra.run.smithi063.stdout:2402 punch from 0x82b9011 to 0x82bcab5, (0x3aa4 bytes) 2024-09-13T09:26:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:2403 writesame 0xe4e864f thru 0xe4eb594 data_size 0x1(0x2f46 bytes) 2024-09-13T09:26:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:2404 write 0x80e7cf4 thru 0x80ec3f6 (0x4703 bytes) 2024-09-13T09:26:49.803 INFO:teuthology.orchestra.run.smithi063.stdout:2405 punch from 0x8af77b5 to 0x8afa3aa, (0x2bf5 bytes) 2024-09-13T09:26:49.809 INFO:teuthology.orchestra.run.smithi063.stdout:2406 trunc from 0xe4eb595 to 0xbc57419 2024-09-13T09:26:49.811 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:49.808+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:49.834 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.835 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:49.856 INFO:teuthology.orchestra.run.smithi063.stdout:2408 write 0x8008a54 thru 0x80109b1 (0x7f5e bytes) 2024-09-13T09:26:49.862 INFO:teuthology.orchestra.run.smithi063.stdout:2409 trunc from 0xbc57419 to 0xccd6b0f 2024-09-13T09:26:49.872 INFO:teuthology.orchestra.run.smithi063.stdout:2410 write 0x9039761 thru 0x9042470 (0x8d10 bytes) 2024-09-13T09:26:49.888 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:49.888 INFO:teuthology.orchestra.run.smithi063.stdout:2412 writesame 0x2f0c27a thru 0x2f18b36 data_size 0x3b(0xc8bd bytes) 2024-09-13T09:26:49.891 INFO:teuthology.orchestra.run.smithi063.stdout:2413 read 0x7e543d thru 0x7ea5e8 (0x51ac bytes) 2024-09-13T09:26:49.899 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:49.903 INFO:teuthology.orchestra.run.smithi063.stdout:2414 read 0x5450e13 thru 0x545538c (0x457a bytes) 2024-09-13T09:26:49.911 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:49.903+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:49.915 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:49.915 INFO:teuthology.orchestra.run.smithi063.stdout:2420 writesame 0x315f682 thru 0x316225a data_size 0x5(0x2bd9 bytes) 2024-09-13T09:26:49.927 INFO:teuthology.orchestra.run.smithi063.stdout:2421 write 0x14f14a3 thru 0x14fcfe7 (0xbb45 bytes) 2024-09-13T09:26:49.935 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-13T09:26:49.935 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:49.935 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:26:49.936 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-13T09:26:49.936 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T09:26:49.976 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:49.976 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:49.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:50.012 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:50.003+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:50.014 INFO:teuthology.orchestra.run.smithi063.stdout:2423 read 0x57b91dd thru 0x57bcdd6 (0x3bfa bytes) 2024-09-13T09:26:50.026 INFO:teuthology.orchestra.run.smithi063.stdout:2425 trunc from 0xccd6b0f to 0x8a375e7 2024-09-13T09:26:50.064 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:50.066 INFO:teuthology.orchestra.run.smithi063.stdout:2426 write 0x28f4e8f thru 0x28f5b31 (0xca3 bytes) 2024-09-13T09:26:50.080 INFO:teuthology.orchestra.run.smithi063.stdout:2429 write 0xa7a26e0 thru 0xa7b1471 (0xed92 bytes) 2024-09-13T09:26:50.086 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:50.093 INFO:teuthology.orchestra.run.smithi063.stdout:2431 trunc from 0xa7b1472 to 0x47f52fe 2024-09-13T09:26:50.101 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:50.101 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:50.110 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:50.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:50.113 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:50.108+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:50.137 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:50.154 INFO:teuthology.orchestra.run.smithi063.stdout:2433 write 0xca96766 thru 0xca9bee0 (0x577b bytes) 2024-09-13T09:26:50.156 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:50.158 INFO:teuthology.orchestra.run.smithi063.stdout:2434 read 0x3841842 thru 0x38422a0 (0xa5f bytes) 2024-09-13T09:26:50.159 INFO:teuthology.orchestra.run.smithi063.stdout:2435 write 0x687b0f5 thru 0x687bdec (0xcf8 bytes) 2024-09-13T09:26:50.161 INFO:teuthology.orchestra.run.smithi063.stdout:2436 read 0x8a2a thru 0x1410d (0xb6e4 bytes) 2024-09-13T09:26:50.162 INFO:teuthology.orchestra.run.smithi063.stdout:2437 writesame 0xc18736 thru 0xc1fa74 data_size 0x1(0x733f bytes) 2024-09-13T09:26:50.181 INFO:teuthology.orchestra.run.smithi063.stdout:2438 read 0xa49fcad thru 0xa4a5f65 (0x62b9 bytes) 2024-09-13T09:26:50.181 INFO:teuthology.orchestra.run.smithi063.stdout:2439 writesame 0x956d3ba thru 0x956dbca data_size 0x1(0x811 bytes) 2024-09-13T09:26:50.183 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:50.187 INFO:teuthology.orchestra.run.smithi063.stdout:2440 write 0x502299b thru 0x502d0fa (0xa760 bytes) 2024-09-13T09:26:50.198 INFO:teuthology.orchestra.run.smithi063.stdout:2442 write 0xd54dff0 thru 0xd5542d4 (0x62e5 bytes) 2024-09-13T09:26:50.202 INFO:teuthology.orchestra.run.smithi063.stdout:2443 write 0x69452e9 thru 0x6947763 (0x247b bytes) 2024-09-13T09:26:50.205 INFO:teuthology.orchestra.run.smithi063.stdout:2444 writesame 0xc972ca9 thru 0xc97f1dc data_size 0x2a(0xc534 bytes) 2024-09-13T09:26:50.210 INFO:teuthology.orchestra.run.smithi063.stdout:2445 read 0x3fa209 thru 0x40387d (0x9675 bytes) 2024-09-13T09:26:50.211 INFO:teuthology.orchestra.run.smithi063.stdout:2448 writesame 0xbc7640d thru 0xbc768e8 data_size 0x1(0x4dc bytes) 2024-09-13T09:26:50.213 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:50.212+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:50.218 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:50.218 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:50.218 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:50.218 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:50.218 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:50.219 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:50.225 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:50.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:50.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:50.231 INFO:teuthology.orchestra.run.smithi063.stdout:2450 trunc from 0xd5542d5 to 0x5b1f9af 2024-09-13T09:26:50.250 INFO:teuthology.orchestra.run.smithi063.stdout:2452 read 0x2cb3221 thru 0x2cb789e (0x467e bytes) 2024-09-13T09:26:50.258 INFO:teuthology.orchestra.run.smithi063.stdout:2453 write 0x5d3ba23 thru 0x5d481dc (0xc7ba bytes) 2024-09-13T09:26:50.261 INFO:teuthology.orchestra.run.smithi063.stdout:2454 punch from 0x2adfd37 to 0x2ae7434, (0x76fd bytes) 2024-09-13T09:26:50.261 INFO:teuthology.orchestra.run.smithi063.stdout:2455 read 0x50d0682 thru 0x50da4b4 (0x9e33 bytes) 2024-09-13T09:26:50.261 INFO:teuthology.orchestra.run.smithi063.stdout:2456 write 0x10d2296 thru 0x10e1370 (0xf0db bytes) 2024-09-13T09:26:50.315 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:50.312+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:50.358 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:50.358 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 67.395242511999996, 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.359 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:50.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:50.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:50.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.282022768, 2024-09-13T09:26:50.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:26:50.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.281624753, 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:26:50.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6065178769999999, 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:50.376 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6065029449999999, 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6064698389999998, 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:26:50.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.606453546, 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:26:50.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.389 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6064374309999998, 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:26:50.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:26:50.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6064063900000001, 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.394 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:26:50.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:26:50.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.6063747670000001, 2024-09-13T09:26:50.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5954814549999998, 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:26:50.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5954611180000002, 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:26:50.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:26:50.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5934267059999998, 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5934096499999999, 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:26:50.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:26:50.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:26:50.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5933934819999998, 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:26:50.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.433 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.59317982, 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:26:50.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5931605090000001, 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:26:50.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5931441159999999, 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:26:50.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:26:50.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5931284539999999, 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:26:50.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5931020999999999, 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:50.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:26:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:26:50.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.459 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:50.459 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:50.459 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:50.461 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:50.456+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:50.462 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:50.463 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.463 INFO:teuthology.orchestra.run.smithi063.stdout:2459 writesame 0x3e0d6ed thru 0x3e0f5e1 data_size 0x5(0x1ef5 bytes) 2024-09-13T09:26:50.463 INFO:teuthology.orchestra.run.smithi063.stdout:2460 read 0x2cd753e thru 0x2cdc1a6 (0x4c69 bytes) 2024-09-13T09:26:50.463 INFO:teuthology.orchestra.run.smithi063.stdout:2461 trunc from 0x5d481dd to 0x8ad41cf 2024-09-13T09:26:50.463 INFO:teuthology.orchestra.run.smithi063.stdout:2462 write 0x9eec170 thru 0x9ef950d (0xd39e bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2465 writesame 0xbcf6485 thru 0xbd03cec data_size 0x32(0xd868 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2466 read 0x2223e9a thru 0x2225f4d (0x20b4 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2467 read 0x1ee81e0 thru 0x1eed728 (0x5549 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2470 read 0x8464845 thru 0x8471dde (0xd59a bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2471 writesame 0xd895506 thru 0xd89ea17 data_size 0x1(0x9512 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2472 read 0xbd3f2ed thru 0xbd44a9e (0x57b2 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2473 punch from 0x6d30b7e to 0x6d39f4f, (0x93d1 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2474 read 0xcd9f759 thru 0xcda61f0 (0x6a98 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2475 write 0xca3826b thru 0xca448bb (0xc651 bytes) 2024-09-13T09:26:50.464 INFO:teuthology.orchestra.run.smithi063.stdout:2476 writesame 0xda1749e thru 0xda1b274 data_size 0x3(0x3dd7 bytes) 2024-09-13T09:26:50.466 INFO:teuthology.orchestra.run.smithi063.stdout:2477 writesame 0x80f6501 thru 0x80fbced data_size 0x3d(0x57ed bytes) 2024-09-13T09:26:50.474 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.474 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.474 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.474 INFO:teuthology.orchestra.run.smithi063.stdout:2482 read 0x6116e69 thru 0x6125264 (0xe3fc bytes) 2024-09-13T09:26:50.475 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.475 INFO:teuthology.orchestra.run.smithi063.stdout:2484 write 0x8d965ce thru 0x8d9d554 (0x6f87 bytes) 2024-09-13T09:26:50.481 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.482 INFO:teuthology.orchestra.run.smithi063.stdout:2486 write 0xa57dbd7 thru 0xa58a959 (0xcd83 bytes) 2024-09-13T09:26:50.490 INFO:teuthology.orchestra.run.smithi063.stdout:2487 trunc from 0xda1b275 to 0xd6e0780 2024-09-13T09:26:50.503 INFO:teuthology.orchestra.run.smithi063.stdout:2490 punch from 0x9df131f to 0x9dfe039, (0xcd1a bytes) 2024-09-13T09:26:50.503 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.503 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.503 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.503 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.503 INFO:teuthology.orchestra.run.smithi063.stdout:2495 read 0x85e4deb thru 0x85efe92 (0xb0a8 bytes) 2024-09-13T09:26:50.522 INFO:teuthology.orchestra.run.smithi063.stdout:2496 trunc from 0xd6e0780 to 0x44f66d1 2024-09-13T09:26:50.548 INFO:teuthology.orchestra.run.smithi063.stdout:2497 read 0x3abefe3 thru 0x3acb130 (0xc14e bytes) 2024-09-13T09:26:50.549 INFO:teuthology.orchestra.run.smithi063.stdout:2498 read 0xb554a5 thru 0xb5b3e1 (0x5f3d bytes) 2024-09-13T09:26:50.550 INFO:teuthology.orchestra.run.smithi063.stdout:2500 punch from 0xddf22 to 0xe6a7a, (0x8b58 bytes) 2024-09-13T09:26:50.561 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:50.556+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:50.563 INFO:teuthology.orchestra.run.smithi063.stdout:2501 trunc from 0x44f66d1 to 0xbb40c68 2024-09-13T09:26:50.572 INFO:teuthology.orchestra.run.smithi063.stdout:2502 read 0xb4d8351 thru 0xb4e1a9a (0x974a bytes) 2024-09-13T09:26:50.572 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.573 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.573 INFO:teuthology.orchestra.run.smithi063.stdout:2507 punch from 0x34029e9 to 0x340ab5e, (0x8175 bytes) 2024-09-13T09:26:50.573 INFO:teuthology.orchestra.run.smithi063.stdout:2508 punch from 0x3c9143b to 0x3c9b651, (0xa216 bytes) 2024-09-13T09:26:50.573 INFO:teuthology.orchestra.run.smithi063.stdout:2509 writesame 0x608879b thru 0x608f556 data_size 0x6(0x6dbc bytes) 2024-09-13T09:26:50.578 INFO:teuthology.orchestra.run.smithi063.stdout:2510 read 0x74ef17e thru 0x74f60c9 (0x6f4c bytes) 2024-09-13T09:26:50.588 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:50.588 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:50.588 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:50.588 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:50.589 INFO:teuthology.orchestra.run.smithi063.stdout:2511 trunc from 0xbb40c68 to 0x607d9ae 2024-09-13T09:26:50.600 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:50.601 INFO:teuthology.orchestra.run.smithi063.stdout:2512 write 0x3010711 thru 0x3014686 (0x3f76 bytes) 2024-09-13T09:26:50.604 INFO:teuthology.orchestra.run.smithi063.stdout:2513 read 0x533ef12 thru 0x5340cf7 (0x1de6 bytes) 2024-09-13T09:26:50.605 INFO:teuthology.orchestra.run.smithi063.stdout:2514 read 0xe14fa8 thru 0xe160ed (0x1146 bytes) 2024-09-13T09:26:50.628 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T09:26:50.630 INFO:teuthology.orchestra.run.smithi063.stdout:2516 writesame 0x6531d64 thru 0x653eb23 data_size 0x20(0xcdc0 bytes) 2024-09-13T09:26:50.633 INFO:teuthology.orchestra.run.smithi063.stdout:2517 read 0x3c7e52d thru 0x3c83db1 (0x5885 bytes) 2024-09-13T09:26:50.635 INFO:teuthology.orchestra.run.smithi063.stdout:2518 punch from 0x3e8443e to 0x3e93afa, (0xf6bc bytes) 2024-09-13T09:26:50.635 INFO:teuthology.orchestra.run.smithi063.stdout:2519 punch from 0x3b29674 to 0x3b2bc4e, (0x25da bytes) 2024-09-13T09:26:50.635 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.636 INFO:teuthology.orchestra.run.smithi063.stdout:2522 read 0x19c62c9 thru 0x19cff8e (0x9cc6 bytes) 2024-09-13T09:26:50.638 INFO:teuthology.orchestra.run.smithi063.stdout:2523 write 0x75235a thru 0x759205 (0x6eac bytes) 2024-09-13T09:26:50.642 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:50.642 INFO:teuthology.orchestra.run.smithi063.stdout:2525 clone 22 order 19 su 32768 sc 14 2024-09-13T09:26:50.662 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:50.660+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:50.706 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:50.707 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:50.707 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:50.707 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:50.707 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:50.716 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:50.763 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:50.755+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:50.827 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:50.827 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.2589757600000002, 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:26:50.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:26:50.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:72716 3.12 3:4f8411ec:::rbd_data.11181839c212.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:26:50.833 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0167347119999999, 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0081225499999999992, 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 72716 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:26:50.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804331+0000", 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804537+0000", 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020520799999999999 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804563+0000", 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6067999999999998e-05 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804575+0000", 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2618e-05 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804679+0000", 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104003 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804949+0000", 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026972500000000001 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.805349+0000", 2024-09-13T09:26:50.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039944600000000002 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.808978+0000", 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036290369999999999 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812305+0000", 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033272900000000001 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812391+0000", 2024-09-13T09:26:50.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6250999999999998e-05 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812456+0000", 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5324e-05 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:77449 3.1 3:8066b423:::rbd_data.11181839c212.0000000000000104:head [set-alloc-hint object_size 262144 write_size 262144,write 225280~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.183422+0000", 2024-09-13T09:26:50.839 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6376465409999996, 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.006692333, 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 77449 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183422+0000", 2024-09-13T09:26:50.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183422+0000", 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183419+0000", 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183440+0000", 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1013999999999999e-05 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183470+0000", 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0790000000000002e-05 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183504+0000", 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3259999999999997e-05 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183625+0000", 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012167099999999999 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.183869+0000", 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000243475 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.184099+0000", 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000229948 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.185636+0000", 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001537011 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.189979+0000", 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043436159999999998 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.190066+0000", 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7051000000000004e-05 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.190114+0000", 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8016999999999998e-05 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:77451 3.12 3:49ce8068:::rbd_data.11181839c212.0000000000000106:head [set-alloc-hint object_size 262144 write_size 262144,write 224614~4762] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.186446+0000", 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6346227530000004, 2024-09-13T09:26:50.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0064509900000000002, 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 77451 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186446+0000", 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186446+0000", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186444+0000", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186466+0000", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2633000000000002e-05 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186527+0000", 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1173999999999994e-05 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186560+0000", 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2134000000000001e-05 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.186723+0000", 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016382199999999999 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.187050+0000", 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032627200000000001 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.187356+0000", 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030606999999999997 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.188849+0000", 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00149361 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.192719+0000", 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038694329999999998 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.192844+0000", 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012503199999999999 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.192897+0000", 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3066999999999999e-05 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:77476 3.1b 3:da9463a3:::rbd_data.11181839c212.0000000000000179:head [set-alloc-hint object_size 262144 write_size 262144,write 99328~3596] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.204410+0000", 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6166590139999997, 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058453369999999999, 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 77476 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204410+0000", 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204410+0000", 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204408+0000", 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204420+0000", 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2624e-05 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204448+0000", 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7246e-05 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204463+0000", 2024-09-13T09:26:50.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5685e-05 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204549+0000", 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5606999999999995e-05 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204725+0000", 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000176391 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.204875+0000", 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014927899999999999 2024-09-13T09:26:50.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.206236+0000", 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013617900000000001 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210025+0000", 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037883249999999999 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210097+0000", 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2675000000000005e-05 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210255+0000", 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015750200000000001 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87135 3.1b 3:d87e56f6:::rbd_data.11181839c212.0000000000000020:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:26:50.856 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.031090711, 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085077469999999995, 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87135 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789975+0000", 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789989+0000", 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790012+0000", 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2889999999999999e-05 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790034+0000", 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2221e-05 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790129+0000", 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5093000000000006e-05 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790376+0000", 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024744900000000001 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790600+0000", 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223654 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.791728+0000", 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011281170000000001 2024-09-13T09:26:50.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798262+0000", 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065342899999999999 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798385+0000", 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012293199999999999 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798486+0000", 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100324 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87179 3.1b 3:db5faf4f:::rbd_data.11181839c212.00000000000001e3:head [write 27648~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.861665+0000", 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.959403579, 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057942769999999996, 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87179 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861665+0000", 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861665+0000", 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861664+0000", 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861674+0000", 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.013e-05 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861698+0000", 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.393e-05 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861713+0000", 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5173e-05 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.861855+0000", 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000142294 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.862073+0000", 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021767600000000001 2024-09-13T09:26:50.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.862259+0000", 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000186007 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.864529+0000", 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022696219999999998 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.867316+0000", 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002787613 2024-09-13T09:26:50.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.867376+0000", 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9460000000000003e-05 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.867459+0000", 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3688000000000002e-05 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.867 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87216 3.1 3:821435ba:::rbd_data.11181839c212.00000000000001db:head [set-alloc-hint object_size 262144 write_size 262144,write 82944~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.883346+0000", 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.93772214, 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0064866330000000003, 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87216 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883346+0000", 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883346+0000", 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883344+0000", 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883369+0000", 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4669e-05 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883407+0000", 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8430999999999998e-05 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883427+0000", 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9729e-05 2024-09-13T09:26:50.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883551+0000", 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012453400000000001 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.883775+0000", 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022318699999999999 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.884102+0000", 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032791100000000001 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.885778+0000", 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016758789999999999 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.889717+0000", 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003938436 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.889783+0000", 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6185000000000004e-05 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.889833+0000", 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0126000000000001e-05 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.873 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87219 3.12 3:4a989717:::rbd_data.11181839c212.00000000000001de:head [set-alloc-hint object_size 262144 write_size 262144,write 82944~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.886073+0000", 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.9349957, 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.006338738, 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87219 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886073+0000", 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886073+0000", 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886071+0000", 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:50.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886089+0000", 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8213999999999999e-05 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886122+0000", 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2911999999999999e-05 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886140+0000", 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8332e-05 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886229+0000", 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8576e-05 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886395+0000", 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016651200000000001 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.886576+0000", 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000180982 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.888286+0000", 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017104539999999999 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.892281+0000", 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00399461 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.892363+0000", 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2089000000000006e-05 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.892412+0000", 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8463e-05 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87239 3.12 3:4ce3797a:::rbd_data.11181839c212.0000000000000133:head [set-alloc-hint object_size 262144 write_size 262144,write 144384~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.879 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.9157115720000002, 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070822869999999996, 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87239 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:26:50.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905355+0000", 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905367+0000", 2024-09-13T09:26:50.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1385e-05 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905394+0000", 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7627000000000002e-05 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905412+0000", 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7076999999999999e-05 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905512+0000", 2024-09-13T09:26:50.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100139 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905698+0000", 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000186532 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905875+0000", 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000176751 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.907019+0000", 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00114375 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912216+0000", 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051968719999999999 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912351+0000", 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013516099999999999 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912439+0000", 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8578000000000005e-05 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87621 3.1b 3:d95b6ede:::rbd_data.11181839c212.000000000000026d:head [set-alloc-hint object_size 262144 write_size 262144,write 24576~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.215387+0000", 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.6056815090000001, 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063115009999999997, 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87621 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215387+0000", 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215387+0000", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215384+0000", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215399+0000", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4995e-05 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215431+0000", 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2682999999999999e-05 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215451+0000", 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9930999999999999e-05 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215578+0000", 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000126802 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.215851+0000", 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000272651 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.216167+0000", 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00031629199999999997 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.217609+0000", 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014417169999999999 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.221565+0000", 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039564079999999998 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.221641+0000", 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6111000000000004e-05 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.221699+0000", 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7317e-05 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.890 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87857 3.b 3:d4eee7ff:::rbd_data.11181839c212.0000000000000197:head [set-alloc-hint object_size 262144 write_size 262144,write 179200~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.427040+0000", 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.394028826, 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058343409999999998, 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87857 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.891 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427040+0000", 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427040+0000", 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427038+0000", 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427059+0000", 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1549999999999999e-05 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427093+0000", 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3457000000000002e-05 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427108+0000", 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.507e-05 2024-09-13T09:26:50.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427200+0000", 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2311000000000004e-05 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427351+0000", 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000150963 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.427515+0000", 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016432400000000001 2024-09-13T09:26:50.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.428616+0000", 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001100796 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.432605+0000", 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039886490000000004 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.432695+0000", 2024-09-13T09:26:50.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0476999999999994e-05 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.432874+0000", 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178841 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87861 3.1b 3:d9678466:::rbd_data.11181839c212.000000000000019b:head [set-alloc-hint object_size 262144 write_size 262144,write 179200~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:26:50.896 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.386963073, 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074187890000000003, 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87861 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434103+0000", 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434118+0000", 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4357e-05 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434154+0000", 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5816999999999999e-05 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434173+0000", 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9497e-05 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434266+0000", 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2987999999999998e-05 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434491+0000", 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000224441 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434675+0000", 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018440899999999999 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.436722+0000", 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020469669999999998 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441372+0000", 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046502059999999996 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441447+0000", 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4356000000000004e-05 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441524+0000", 2024-09-13T09:26:50.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7818000000000006e-05 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87915 3.12 3:4c727ac7:::rbd_data.11181839c212.00000000000000b4:head [set-alloc-hint object_size 262144 write_size 262144,write 53248~2119] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.453471+0000", 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.3675972789999999, 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063088759999999997, 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.902 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87915 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453471+0000", 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453471+0000", 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453470+0000", 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453483+0000", 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3896999999999999e-05 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453502+0000", 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8698e-05 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453510+0000", 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2670000000000006e-06 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453580+0000", 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9525999999999998e-05 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453721+0000", 2024-09-13T09:26:50.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014112099999999999 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.453839+0000", 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118027 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.454721+0000", 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00088180500000000005 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.459467+0000", 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047458220000000002 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.459649+0000", 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018196299999999999 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.459780+0000", 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000131543 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.907 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87976 3.1b 3:dfd74b2f:::rbd_data.11181839c212.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.2883634719999999, 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074366880000000003, 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87976 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.908 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532703+0000", 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532716+0000", 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532741+0000", 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5565999999999999e-05 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532756+0000", 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4367e-05 2024-09-13T09:26:50.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532900+0000", 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014460699999999999 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533180+0000", 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027956200000000002 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533377+0000", 2024-09-13T09:26:50.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019749399999999999 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535597+0000", 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022201439999999999 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.539992+0000", 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043942759999999999 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540079+0000", 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6942999999999997e-05 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540142+0000", 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3192000000000001e-05 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87977 3.1 3:83505842:::rbd_data.11181839c212.000000000000027b:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.913 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.2882614889999999, 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074050050000000001, 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87977 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532806+0000", 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532815+0000", 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.346e-06 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532844+0000", 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8484e-05 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532853+0000", 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9700000000000005e-06 2024-09-13T09:26:50.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532953+0000", 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100234 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533161+0000", 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020798600000000001 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533328+0000", 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016667299999999999 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535098+0000", 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017702589999999999 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540105+0000", 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050068170000000002 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540178+0000", 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3165999999999997e-05 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540212+0000", 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4175999999999999e-05 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87980 3.1b 3:dacf50f8:::rbd_data.11181839c212.000000000000027e:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.919 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.536780+0000", 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.2842890300000001, 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059932049999999997, 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87980 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536780+0000", 2024-09-13T09:26:50.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536780+0000", 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536776+0000", 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536791+0000", 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.484e-05 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536831+0000", 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9796000000000003e-05 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536847+0000", 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5497000000000001e-05 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.536994+0000", 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014709099999999999 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.537252+0000", 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025826400000000002 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.537502+0000", 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024992200000000002 2024-09-13T09:26:50.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.538904+0000", 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001402492 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.542654+0000", 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037498520000000001 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.542738+0000", 2024-09-13T09:26:50.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3844000000000006e-05 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.542773+0000", 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4823000000000002e-05 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:88074 3.12 3:4fffb25f:::rbd_data.11181839c212.0000000000000230:head [set-alloc-hint object_size 262144 write_size 262144,write 55296~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.635286+0000", 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.1857829560000002, 2024-09-13T09:26:50.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058715, 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 88074 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635286+0000", 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635286+0000", 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635284+0000", 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635295+0000", 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1307e-05 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635316+0000", 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0302e-05 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635326+0000", 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0067e-05 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635394+0000", 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8585999999999997e-05 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635624+0000", 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022986099999999999 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.635823+0000", 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019855799999999999 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.637172+0000", 2024-09-13T09:26:50.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001349818 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.641044+0000", 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038720820000000002 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.641114+0000", 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9784999999999997e-05 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.641157+0000", 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2888e-05 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.65975166699999999, 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:26:50.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:26:50.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.936 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.55140647799999998, 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:26:50.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:26:50.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:50.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:26:50.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:50.942 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:50.944 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:50.944 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:50.939+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:51.045 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:51.040+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:51.059 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:51.060 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:51.060 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:51.060 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:51.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:51.146 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:51.144+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:51.177 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:51.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:51.177 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:51.177 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:51.178 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:51.187 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:51.246 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:51.239+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:26:51.301 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.342367310999997, 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:26:51.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.649954671, 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:26:51.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.649834109, 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:26:51.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:26:51.315 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.177060925999999, 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.316 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.321 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7467079300000004, 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:26:51.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.325 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.746682056, 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.326 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7466566920000002, 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:26:51.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:26:51.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7465716699999998, 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:26:51.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:26:51.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:26:51.338 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.339 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.339 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.339 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.339 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7465382749999998, 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:26:51.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.343 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7465243370000003, 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:26:51.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.746486591, 2024-09-13T09:26:51.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:26:51.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.352 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.746468675, 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:26:51.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:26:51.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:26:51.357 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7464550269999997, 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:26:51.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7463883820000001, 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.362 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:26:51.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:26:51.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.746340322, 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.367 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7462830499999997, 2024-09-13T09:26:51.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:26:51.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.377 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7462629410000003, 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.378 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:26:51.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7462407950000003, 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:51.383 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:26:51.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7461845279999997, 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:26:51.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7461686319999998, 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:51.393 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:26:51.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.397 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:51.399 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:51.399 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:51.395+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:51.499 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:51.491+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:51.557 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:51.557 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:51.557 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:51.557 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:51.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:51.600 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:51.596+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:51.635 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:26:51.685 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:51.685 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:51.685 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:51.685 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:51.685 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:51.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:51.700 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:51.691+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:51.802 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:51.795+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:51.805 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone21 from 0x653eb24 (overlap 0x44f66d1) to 0x0 2024-09-13T09:26:51.812 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:276 3.4 e27/21 3:27927849:::rbd_data.111867511a81.0000000000000026:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.302144544000001, 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002929961, 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504324+0000", 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504366+0000", 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2412e-05 2024-09-13T09:26:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504407+0000", 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0329e-05 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504427+0000", 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0089999999999999e-05 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504518+0000", 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0801e-05 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504566+0000", 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8745999999999997e-05 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507192+0000", 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00262546 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507256+0000", 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.003531549000002, 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:26:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:26:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:26:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 48.595355664000003, 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:26:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 48.595165541, 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:26:51.827 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:26:51.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 48.594353132000002, 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.832 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:26:51.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 46.405446482999999, 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:51.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:26:51.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.840337097999999, 2024-09-13T09:26:51.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:26:51.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.766798084999998, 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:26:51.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:26:51.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:26:51.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:51.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:26:51.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:26:51.853 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:51.854 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.674254716999997, 2024-09-13T09:26:51.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:26:51.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.854 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.854 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:51.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:26:51.859 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.646194140999999, 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:26:51.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.631936358000001, 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:26:51.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:26:51.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.869 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.163217905, 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:26:51.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.874 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.174293212, 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:26:51.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.969267164, 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:26:51.879 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.870929115999999, 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.884 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:26:51.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.870120511, 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:26:51.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:26:51.893 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.7955279409999996, 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:26:51.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.6014454980000004, 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:26:51.898 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:26:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.485823999, 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.903 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:26:51.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:26:51.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.2170826569999997, 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:51.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:26:51.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:51.912 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:51.914 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:51.914 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:51.907+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:52.008 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T09:26:52.015 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:52.007+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96207 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023610+0000", 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.02954263, 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029567564000000001, 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96207 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023610+0000", 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023610+0000", 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023610+0000", 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023612+0000", 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4320000000000002e-06 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023618+0000", 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6380000000000001e-06 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023623+0000", 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.848e-06 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96706 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052265+0000", 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00088737799999999997, 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099272199999999992, 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96706 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052265+0000", 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052265+0000", 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052265+0000", 2024-09-13T09:26:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052267+0000", 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052272+0000", 2024-09-13T09:26:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.118e-06 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052277+0000", 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1880000000000001e-06 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96738 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052856+0000", 2024-09-13T09:26:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00029634000000000003, 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045347699999999998, 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96738 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052856+0000", 2024-09-13T09:26:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052856+0000", 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052856+0000", 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052858+0000", 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5040000000000001e-06 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052864+0000", 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6099999999999997e-06 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052869+0000", 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0300000000000001e-06 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96208 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023745+0000", 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029407889, 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029630683000000001, 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96208 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023745+0000", 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023745+0000", 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023744+0000", 2024-09-13T09:26:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023748+0000", 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.72e-06 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023755+0000", 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2629999999999997e-06 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023761+0000", 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1589999999999998e-06 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96406 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.038709+0000", 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014443654, 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014717225, 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96406 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038709+0000", 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038709+0000", 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038708+0000", 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038714+0000", 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2689999999999999e-06 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038724+0000", 2024-09-13T09:26:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0512e-05 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038732+0000", 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9209999999999992e-06 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96523 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.043875+0000", 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0092772730000000008, 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096031120000000005, 2024-09-13T09:26:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96523 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043875+0000", 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043875+0000", 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043875+0000", 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043877+0000", 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:26:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043882+0000", 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043887+0000", 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.668e-06 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96555 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049070+0000", 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0040822690000000003, 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044584999999999998, 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96555 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049070+0000", 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049070+0000", 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049070+0000", 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049072+0000", 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5239999999999999e-06 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049078+0000", 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049082+0000", 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5050000000000001e-06 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96707 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052282+0000", 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00087084900000000004, 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00131661, 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96707 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052282+0000", 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052282+0000", 2024-09-13T09:26:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052281+0000", 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052284+0000", 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.638e-06 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052290+0000", 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4670000000000003e-06 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052294+0000", 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5730000000000001e-06 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96739 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052873+0000", 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00027956200000000002, 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077464799999999996, 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96739 2024-09-13T09:26:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052873+0000", 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052873+0000", 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052873+0000", 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052875+0000", 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2819999999999999e-06 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052881+0000", 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7409999999999998e-06 2024-09-13T09:26:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052886+0000", 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8330000000000001e-06 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96209 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023766+0000", 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029386716, 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029940614000000001, 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96209 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023766+0000", 2024-09-13T09:26:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023766+0000", 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023766+0000", 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023768+0000", 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023774+0000", 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7740000000000001e-06 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023779+0000", 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0440000000000003e-06 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96708 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052298+0000", 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00085444899999999996, 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014607789999999999, 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96708 2024-09-13T09:26:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052298+0000", 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052298+0000", 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052298+0000", 2024-09-13T09:26:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052300+0000", 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052307+0000", 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052311+0000", 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.775e-06 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96740 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052891+0000", 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00026197200000000002, 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00091918700000000002, 2024-09-13T09:26:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96740 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052891+0000", 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052891+0000", 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052890+0000", 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052893+0000", 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3999999999999999e-06 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052898+0000", 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5130000000000004e-06 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052903+0000", 2024-09-13T09:26:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6600000000000003e-06 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96210 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023783+0000", 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029369323999999999, 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030095957999999999, 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96210 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023783+0000", 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023783+0000", 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023783+0000", 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023786+0000", 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.982e-06 2024-09-13T09:26:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023792+0000", 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8030000000000002e-06 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023796+0000", 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96408 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.038757+0000", 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.014395768999999999, 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015175139000000001, 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96408 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038757+0000", 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038757+0000", 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038756+0000", 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038760+0000", 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8160000000000004e-06 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038766+0000", 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.038771+0000", 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.057e-06 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96525 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.043922+0000", 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.009230617, 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010061109, 2024-09-13T09:26:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96525 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043922+0000", 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043922+0000", 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043922+0000", 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043924+0000", 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:26:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043929+0000", 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1819999999999997e-06 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043934+0000", 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5360000000000003e-06 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96557 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049102+0000", 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0040501249999999999, 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049416080000000001, 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96557 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049102+0000", 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049102+0000", 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049102+0000", 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049105+0000", 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049110+0000", 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3720000000000004e-06 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049115+0000", 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5140000000000003e-06 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96709 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052316+0000", 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00083711199999999999, 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017783269999999999, 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96709 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052316+0000", 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052316+0000", 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052315+0000", 2024-09-13T09:26:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052318+0000", 2024-09-13T09:26:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7099999999999999e-06 2024-09-13T09:26:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052323+0000", 2024-09-13T09:26:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1560000000000003e-06 2024-09-13T09:26:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052327+0000", 2024-09-13T09:26:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2429999999999999e-06 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96741 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052907+0000", 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.000245923, 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012473269999999999, 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96741 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052907+0000", 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052907+0000", 2024-09-13T09:26:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052906+0000", 2024-09-13T09:26:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052922+0000", 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5392000000000001e-05 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052930+0000", 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0779999999999996e-06 2024-09-13T09:26:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052936+0000", 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8640000000000001e-06 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96211 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023801+0000", 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029351806000000001, 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030427645, 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96211 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023801+0000", 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023801+0000", 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023800+0000", 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023803+0000", 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6170000000000001e-06 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023809+0000", 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0009999999999998e-06 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023814+0000", 2024-09-13T09:26:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9350000000000002e-06 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96559 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049119+0000", 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0040332620000000001, 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051626909999999996, 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96559 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049119+0000", 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049119+0000", 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049119+0000", 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049121+0000", 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049126+0000", 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1159999999999998e-06 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049132+0000", 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.993e-06 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96710 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052331+0000", 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00082144599999999998, 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002007886, 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96710 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052331+0000", 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052331+0000", 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052331+0000", 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052333+0000", 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6110000000000001e-06 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052339+0000", 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.728e-06 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052345+0000", 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.57e-06 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96742 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052941+0000", 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00021154, 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014708449999999999, 2024-09-13T09:26:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96742 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052941+0000", 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052941+0000", 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052940+0000", 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052944+0000", 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1549999999999999e-06 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052949+0000", 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3430000000000003e-06 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052954+0000", 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7639999999999996e-06 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96212 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023818+0000", 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029334195, 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030638228999999999, 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96212 2024-09-13T09:26:52.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023818+0000", 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023818+0000", 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023818+0000", 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023821+0000", 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7480000000000002e-06 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023824+0000", 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.394e-06 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023827+0000", 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2669999999999998e-06 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96527 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.043953+0000", 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.009199746, 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010558130000000001, 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96527 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043953+0000", 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043953+0000", 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043953+0000", 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043955+0000", 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.357e-06 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043960+0000", 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2839999999999997e-06 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044464+0000", 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00050345299999999995 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96560 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049137+0000", 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0040157530000000004, 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054083259999999998, 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96560 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049137+0000", 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049137+0000", 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049137+0000", 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049139+0000", 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7310000000000002e-06 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049145+0000", 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6019999999999999e-06 2024-09-13T09:26:52.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049150+0000", 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9439999999999996e-06 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96711 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052349+0000", 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00080339500000000004, 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022402530000000002, 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96711 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052349+0000", 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052349+0000", 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052349+0000", 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052351+0000", 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052357+0000", 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5720000000000002e-06 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052361+0000", 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8040000000000001e-06 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96743 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052959+0000", 2024-09-13T09:26:52.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.000193683, 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016789400000000001, 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96743 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052959+0000", 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052959+0000", 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052958+0000", 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052961+0000", 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7839999999999999e-06 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052969+0000", 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052974+0000", 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96213 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023830+0000", 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029322642999999999, 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030860377000000001, 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96213 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023830+0000", 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023830+0000", 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023830+0000", 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023831+0000", 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5799999999999999e-06 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023834+0000", 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.021e-06 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023837+0000", 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.773e-06 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96562 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049154+0000", 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003998489, 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055977500000000003, 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96562 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049154+0000", 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049154+0000", 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049154+0000", 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049162+0000", 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9359999999999999e-06 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049168+0000", 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049173+0000", 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0429999999999998e-06 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.170 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96712 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052377+0000", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00077528199999999997, 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024578270000000001, 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96712 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052377+0000", 2024-09-13T09:26:52.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052377+0000", 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052377+0000", 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052380+0000", 2024-09-13T09:26:52.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052386+0000", 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4590000000000003e-06 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052391+0000", 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2519999999999999e-06 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96744 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052978+0000", 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.000174799, 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001917261, 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96744 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052978+0000", 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052978+0000", 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052977+0000", 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052980+0000", 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.616e-06 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052986+0000", 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052991+0000", 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1959999999999999e-06 2024-09-13T09:26:52.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96214 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023839+0000", 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029313290999999998, 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031119936000000001, 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96214 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023839+0000", 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023839+0000", 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023839+0000", 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023841+0000", 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6169999999999999e-06 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023844+0000", 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023847+0000", 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96529 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.044490+0000", 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0086624779999999995, 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010535912999999999, 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96529 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044490+0000", 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044490+0000", 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044490+0000", 2024-09-13T09:26:52.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044492+0000", 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4849999999999999e-06 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044497+0000", 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1919999999999996e-06 2024-09-13T09:26:52.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044502+0000", 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5909999999999998e-06 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96564 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049177+0000", 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0039751880000000002, 2024-09-13T09:26:52.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059060229999999998, 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96564 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049177+0000", 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049177+0000", 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049177+0000", 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049180+0000", 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6460000000000002e-06 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049193+0000", 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3020000000000001e-05 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049204+0000", 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1205e-05 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96713 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052396+0000", 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00075648899999999995, 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027374069999999999, 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96713 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052396+0000", 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052396+0000", 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052396+0000", 2024-09-13T09:26:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052398+0000", 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6900000000000001e-06 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052404+0000", 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5269999999999998e-06 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052410+0000", 2024-09-13T09:26:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:26:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96745 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052995+0000", 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00015797599999999999, 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022403359999999999, 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96745 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052995+0000", 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052995+0000", 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052994+0000", 2024-09-13T09:26:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052997+0000", 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053002+0000", 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2850000000000002e-06 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053007+0000", 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7450000000000003e-06 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96413 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.040337+0000", 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.012816084, 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014950909, 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96413 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040337+0000", 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040337+0000", 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040336+0000", 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040341+0000", 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040347+0000", 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040353+0000", 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96565 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049209+0000", 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0039436000000000002, 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061330409999999997, 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96565 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049209+0000", 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049209+0000", 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049208+0000", 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049212+0000", 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.495e-06 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049218+0000", 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5010000000000001e-06 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049888+0000", 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00066933800000000001 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96714 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052415+0000", 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00073780199999999997, 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029732460000000001, 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96714 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052415+0000", 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052415+0000", 2024-09-13T09:26:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052414+0000", 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052417+0000", 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.526e-06 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052424+0000", 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3019999999999997e-06 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052429+0000", 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6550000000000003e-06 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96746 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053011+0000", 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00014168300000000001, 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002415352, 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96746 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053011+0000", 2024-09-13T09:26:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053011+0000", 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053011+0000", 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053013+0000", 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.255e-06 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053018+0000", 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5459999999999999e-06 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053024+0000", 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96531 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.044521+0000", 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0086311770000000003, 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010947146, 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96531 2024-09-13T09:26:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044521+0000", 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044521+0000", 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044521+0000", 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044524+0000", 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044529+0000", 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2920000000000003e-06 2024-09-13T09:26:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044533+0000", 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96567 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049895+0000", 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0032577230000000001, 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056372779999999999, 2024-09-13T09:26:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96567 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049895+0000", 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049895+0000", 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049894+0000", 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049899+0000", 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4710000000000001e-06 2024-09-13T09:26:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049905+0000", 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049910+0000", 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96715 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052433+0000", 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00071988399999999995, 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031536250000000002, 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96715 2024-09-13T09:26:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052433+0000", 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052433+0000", 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052432+0000", 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052435+0000", 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052440+0000", 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3759999999999999e-06 2024-09-13T09:26:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052445+0000", 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6709999999999998e-06 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96747 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053060+0000", 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2362999999999996e-05, 2024-09-13T09:26:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002582282, 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96747 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053060+0000", 2024-09-13T09:26:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053060+0000", 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053060+0000", 2024-09-13T09:26:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053064+0000", 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.177e-06 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053074+0000", 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7799999999999995e-06 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053081+0000", 2024-09-13T09:26:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3640000000000001e-06 2024-09-13T09:26:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96569 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049916+0000", 2024-09-13T09:26:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003236399, 2024-09-13T09:26:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057721379999999996, 2024-09-13T09:26:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96569 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049916+0000", 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049916+0000", 2024-09-13T09:26:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049916+0000", 2024-09-13T09:26:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049919+0000", 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049924+0000", 2024-09-13T09:26:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5740000000000003e-06 2024-09-13T09:26:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049929+0000", 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96716 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052449+0000", 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00070400499999999995, 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032655789999999998, 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96716 2024-09-13T09:26:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052449+0000", 2024-09-13T09:26:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052449+0000", 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052448+0000", 2024-09-13T09:26:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052450+0000", 2024-09-13T09:26:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:26:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052456+0000", 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1889999999999998e-06 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052460+0000", 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3109999999999999e-06 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96748 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053100+0000", 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.3022000000000002e-05, 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026416510000000001, 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96748 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053100+0000", 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053100+0000", 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053099+0000", 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053102+0000", 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9849999999999998e-06 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053106+0000", 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4839999999999998e-06 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053111+0000", 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3810000000000001e-06 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96533 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.044555+0000", 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0085976790000000004, 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011211786, 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96533 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044555+0000", 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044555+0000", 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044555+0000", 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044557+0000", 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:26:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044562+0000", 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5049999999999998e-06 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044567+0000", 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96571 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049934+0000", 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0032190220000000002, 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058596209999999998, 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96571 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049934+0000", 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049934+0000", 2024-09-13T09:26:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049933+0000", 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049936+0000", 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6680000000000001e-06 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049942+0000", 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5459999999999997e-06 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049947+0000", 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3000000000000003e-06 2024-09-13T09:26:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96717 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052469+0000", 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00068411899999999996, 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033540509999999998, 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96717 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052469+0000", 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052469+0000", 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052468+0000", 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052471+0000", 2024-09-13T09:26:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7980000000000001e-06 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052477+0000", 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052482+0000", 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0180000000000001e-06 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96749 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053115+0000", 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.8044999999999999e-05, 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027501700000000001, 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96749 2024-09-13T09:26:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053115+0000", 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053115+0000", 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053114+0000", 2024-09-13T09:26:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053116+0000", 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053121+0000", 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3239999999999996e-06 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053126+0000", 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2780000000000001e-06 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96417 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.040987+0000", 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.012165956, 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014936902, 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96417 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040987+0000", 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040987+0000", 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040986+0000", 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040990+0000", 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.104e-06 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.040997+0000", 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7710000000000001e-06 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041003+0000", 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0870000000000003e-06 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96573 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049950+0000", 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0032021630000000001, 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060279849999999996, 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96573 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049950+0000", 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049950+0000", 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049950+0000", 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049952+0000", 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.385e-06 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049958+0000", 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5790000000000003e-06 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049962+0000", 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3030000000000001e-06 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96718 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052487+0000", 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00066597100000000001, 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035383490000000001, 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96718 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052487+0000", 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052487+0000", 2024-09-13T09:26:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052486+0000", 2024-09-13T09:26:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052489+0000", 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052494+0000", 2024-09-13T09:26:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:26:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052499+0000", 2024-09-13T09:26:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:26:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96750 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053129+0000", 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.3161999999999999e-05, 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002951389, 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96750 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053129+0000", 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053129+0000", 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053129+0000", 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053131+0000", 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9269999999999999e-06 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053135+0000", 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.301e-06 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053139+0000", 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8149999999999999e-06 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96423 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.041008+0000", 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.012144274, 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015129369, 2024-09-13T09:26:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96423 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041008+0000", 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041008+0000", 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041008+0000", 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041011+0000", 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8389999999999998e-06 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041017+0000", 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9730000000000002e-06 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.041022+0000", 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9599999999999999e-06 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96535 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.044587+0000", 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0085659429999999995, 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011600655, 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96535 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044587+0000", 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044587+0000", 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044586+0000", 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044589+0000", 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2929999999999999e-06 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044595+0000", 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1160000000000004e-06 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044599+0000", 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96575 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049966+0000", 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.003186122, 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0062732669999999999, 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96575 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049966+0000", 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049966+0000", 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049966+0000", 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049969+0000", 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049975+0000", 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.173e-06 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049980+0000", 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.797e-06 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96719 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052503+0000", 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00064915600000000002, 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037956380000000001, 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96719 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052503+0000", 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052503+0000", 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052503+0000", 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052505+0000", 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2220000000000001e-06 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052511+0000", 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6690000000000003e-06 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052515+0000", 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3440000000000003e-06 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96751 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053143+0000", 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8009999999999998e-06, 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032101669999999999, 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96751 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053143+0000", 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053143+0000", 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053143+0000", 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053145+0000", 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9580000000000001e-06 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053149+0000", 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053153+0000", 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7740000000000002e-06 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96576 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049984+0000", 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0031689309999999998, 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064181280000000004, 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96576 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049984+0000", 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049984+0000", 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049983+0000", 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049986+0000", 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4710000000000001e-06 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049991+0000", 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4020000000000001e-06 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049996+0000", 2024-09-13T09:26:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96720 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052520+0000", 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00063308899999999998, 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039449289999999998, 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96720 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052520+0000", 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052520+0000", 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052519+0000", 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052522+0000", 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4310000000000001e-06 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052527+0000", 2024-09-13T09:26:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.823e-06 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052532+0000", 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7169999999999999e-06 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96752 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053157+0000", 2024-09-13T09:26:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9999957, 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033579030000000002, 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96752 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053157+0000", 2024-09-13T09:26:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053157+0000", 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053157+0000", 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053159+0000", 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.187e-06 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053164+0000", 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.011e-06 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053168+0000", 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0879999999999997e-06 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96537 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.044636+0000", 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0085167760000000002, 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011931536, 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96537 2024-09-13T09:26:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044636+0000", 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044636+0000", 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044635+0000", 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044638+0000", 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044643+0000", 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0810000000000001e-06 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044647+0000", 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6190000000000002e-06 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96577 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.051642+0000", 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001510714, 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049850770000000001, 2024-09-13T09:26:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96577 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051642+0000", 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051642+0000", 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051641+0000", 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051646+0000", 2024-09-13T09:26:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051909+0000", 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026351 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051916+0000", 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2720000000000003e-06 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96721 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052546+0000", 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00060649, 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004133795, 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96721 2024-09-13T09:26:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052546+0000", 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052546+0000", 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052546+0000", 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052548+0000", 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.976e-06 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052555+0000", 2024-09-13T09:26:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052560+0000", 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9139999999999999e-06 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96754 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053171+0000", 2024-09-13T09:26:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9999814, 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035580020000000002, 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96754 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053171+0000", 2024-09-13T09:26:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053171+0000", 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053171+0000", 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053179+0000", 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7309999999999993e-06 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053184+0000", 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9339999999999997e-06 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053190+0000", 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2210000000000002e-06 2024-09-13T09:26:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96578 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.051922+0000", 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001231029, 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0048575409999999999, 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96578 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051922+0000", 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051922+0000", 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051921+0000", 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051925+0000", 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6600000000000001e-06 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051929+0000", 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5650000000000003e-06 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051932+0000", 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8339999999999999e-06 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96722 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052564+0000", 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00058861899999999997, 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042676140000000003, 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96722 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052564+0000", 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052564+0000", 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052564+0000", 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052566+0000", 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3609999999999999e-06 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052571+0000", 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.322e-06 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052576+0000", 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8080000000000004e-06 2024-09-13T09:26:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96755 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053228+0000", 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9999247, 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036633640000000001, 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96755 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053228+0000", 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053228+0000", 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053227+0000", 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053231+0000", 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4479999999999999e-06 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053237+0000", 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.905e-06 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053243+0000", 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.806e-06 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96539 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.044844+0000", 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0083083250000000001, 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012180046999999999, 2024-09-13T09:26:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96539 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044844+0000", 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044844+0000", 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044844+0000", 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.044847+0000", 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8070000000000001e-06 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046542+0000", 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016949879999999999 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046551+0000", 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.259e-06 2024-09-13T09:26:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96580 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.051936+0000", 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001216738, 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051384880000000001, 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96580 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051936+0000", 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051936+0000", 2024-09-13T09:26:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051936+0000", 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051938+0000", 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9870000000000002e-06 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051940+0000", 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7590000000000002e-06 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051943+0000", 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96723 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052580+0000", 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00057219599999999997, 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045565099999999997, 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96723 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052580+0000", 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052580+0000", 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052580+0000", 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052583+0000", 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052588+0000", 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3550000000000004e-06 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052592+0000", 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96757 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053249+0000", 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9999042, 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039401059999999996, 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96757 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053249+0000", 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053249+0000", 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053248+0000", 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053251+0000", 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:26:52.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053256+0000", 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3349999999999997e-06 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053261+0000", 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7550000000000002e-06 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96684 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.051945+0000", 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0012071530000000001, 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053026319999999998, 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96684 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.343 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051945+0000", 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051945+0000", 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051945+0000", 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051969+0000", 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4017000000000001e-05 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051976+0000", 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.051981+0000", 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.977e-06 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96724 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052597+0000", 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00055526999999999996, 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047100600000000003, 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96724 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052597+0000", 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052597+0000", 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052597+0000", 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052599+0000", 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3319999999999999e-06 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052605+0000", 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8080000000000001e-06 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052609+0000", 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3189999999999996e-06 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.349 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96759 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053265+0000", 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9998879, 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040972370000000001, 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96759 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053265+0000", 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053265+0000", 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053264+0000", 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053267+0000", 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2060000000000001e-06 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053274+0000", 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8850000000000001e-06 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053278+0000", 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6949999999999999e-06 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96541 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.046768+0000", 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0063842309999999998, 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010643998999999999, 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96541 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046768+0000", 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046768+0000", 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046768+0000", 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046772+0000", 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2540000000000003e-06 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046779+0000", 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1509999999999998e-06 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046785+0000", 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0950000000000001e-06 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96688 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052010+0000", 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0011421459999999999, 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054542510000000002, 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96688 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052010+0000", 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052010+0000", 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052010+0000", 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052014+0000", 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4490000000000001e-06 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052023+0000", 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9930000000000001e-06 2024-09-13T09:26:52.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052029+0000", 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2609999999999999e-06 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96725 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052613+0000", 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00053924500000000005, 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049013800000000003, 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96725 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052613+0000", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052613+0000", 2024-09-13T09:26:52.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052613+0000", 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052616+0000", 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6369999999999999e-06 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052621+0000", 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5330000000000002e-06 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052626+0000", 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96761 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053282+0000", 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9998703, 2024-09-13T09:26:52.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042927590000000002, 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96761 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053282+0000", 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053282+0000", 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053282+0000", 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053284+0000", 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.26e-06 2024-09-13T09:26:52.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053290+0000", 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053294+0000", 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6040000000000003e-06 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96689 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052035+0000", 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001117533, 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055933119999999996, 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96689 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052035+0000", 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052035+0000", 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052035+0000", 2024-09-13T09:26:52.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052037+0000", 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.745e-06 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052043+0000", 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4500000000000003e-06 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052047+0000", 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5929999999999999e-06 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96726 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052630+0000", 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00052255800000000001, 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050479640000000003, 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96726 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052630+0000", 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052630+0000", 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052630+0000", 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052632+0000", 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.345e-06 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052639+0000", 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3769999999999998e-06 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052645+0000", 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8019999999999997e-06 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96763 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053298+0000", 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9998546, 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044351629999999998, 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96763 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053298+0000", 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053298+0000", 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053298+0000", 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053300+0000", 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.103e-06 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053305+0000", 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053310+0000", 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6820000000000002e-06 2024-09-13T09:26:52.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96543 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.046963+0000", 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0061892680000000004, 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010832678, 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96543 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046963+0000", 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046963+0000", 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046963+0000", 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.046967+0000", 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1320000000000004e-06 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048625+0000", 2024-09-13T09:26:52.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587419999999999 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048632+0000", 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8630000000000002e-06 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96691 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052052+0000", 2024-09-13T09:26:52.379 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0011001959999999999, 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057799729999999999, 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96691 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052052+0000", 2024-09-13T09:26:52.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052052+0000", 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052052+0000", 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052054+0000", 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5979999999999999e-06 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052060+0000", 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052064+0000", 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.464e-06 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96727 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052650+0000", 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00050268700000000003, 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0052196109999999999, 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96727 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.383 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052650+0000", 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052650+0000", 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052649+0000", 2024-09-13T09:26:52.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052652+0000", 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7080000000000002e-06 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052657+0000", 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3600000000000004e-06 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052662+0000", 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4739999999999999e-06 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96765 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.053314+0000", 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4294967295.9998388, 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045940299999999998, 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96765 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053314+0000", 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053314+0000", 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053314+0000", 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053316+0000", 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053321+0000", 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.053326+0000", 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5909999999999998e-06 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.389 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96693 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052068+0000", 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00108435, 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058870090000000003, 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96693 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052068+0000", 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052068+0000", 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052068+0000", 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052070+0000", 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.17e-06 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052075+0000", 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052080+0000", 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7190000000000001e-06 2024-09-13T09:26:52.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96728 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052667+0000", 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.000485649, 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005341686, 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96728 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052667+0000", 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052667+0000", 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052667+0000", 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052669+0000", 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5330000000000001e-06 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052676+0000", 2024-09-13T09:26:52.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052680+0000", 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.566e-06 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96545 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.048679+0000", 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0044734830000000003, 2024-09-13T09:26:52.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093567779999999996, 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96545 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048679+0000", 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048679+0000", 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048678+0000", 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048682+0000", 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3830000000000001e-06 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048689+0000", 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048694+0000", 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9810000000000003e-06 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.399 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96695 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052084+0000", 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0010685250000000001, 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059792730000000002, 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96695 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052084+0000", 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052084+0000", 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052084+0000", 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052086+0000", 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0870000000000001e-06 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052091+0000", 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1869999999999996e-06 2024-09-13T09:26:52.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052096+0000", 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5120000000000002e-06 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96729 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052684+0000", 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.000468345, 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005423654, 2024-09-13T09:26:52.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96729 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052684+0000", 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052684+0000", 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052684+0000", 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052690+0000", 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4729999999999997e-06 2024-09-13T09:26:52.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052697+0000", 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2890000000000003e-06 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052701+0000", 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5650000000000003e-06 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96697 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052100+0000", 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001052685, 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060570260000000001, 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96697 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.407 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052100+0000", 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052100+0000", 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052100+0000", 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052102+0000", 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.39e-06 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052107+0000", 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.164e-06 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052112+0000", 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7899999999999999e-06 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96730 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052707+0000", 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00044571500000000001, 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054916339999999996, 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.410 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96730 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052707+0000", 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052707+0000", 2024-09-13T09:26:52.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052706+0000", 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052709+0000", 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5940000000000001e-06 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052715+0000", 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8359999999999997e-06 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052720+0000", 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.096e-06 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96547 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.413 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.048714+0000", 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.004438422, 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095116869999999996, 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96547 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048714+0000", 2024-09-13T09:26:52.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048714+0000", 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048714+0000", 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048717+0000", 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048721+0000", 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048726+0000", 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3959999999999999e-06 2024-09-13T09:26:52.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96731 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052724+0000", 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00042848300000000001, 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055304509999999996, 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96731 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052724+0000", 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052724+0000", 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052724+0000", 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052727+0000", 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.898e-06 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052734+0000", 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3159999999999999e-06 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052739+0000", 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.843e-06 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96700 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052132+0000", 2024-09-13T09:26:52.420 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.001020392, 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0061546159999999999, 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96700 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052132+0000", 2024-09-13T09:26:52.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052132+0000", 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052132+0000", 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052134+0000", 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.069e-06 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052139+0000", 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8350000000000003e-06 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052143+0000", 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8519999999999997e-06 2024-09-13T09:26:52.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96732 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052753+0000", 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00039978099999999999, 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055789560000000004, 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96732 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052753+0000", 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052753+0000", 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052752+0000", 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052755+0000", 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9730000000000002e-06 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052762+0000", 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1359999999999999e-06 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052768+0000", 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6230000000000002e-06 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96549 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.048765+0000", 2024-09-13T09:26:52.427 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0043877070000000002, 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00960424, 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96549 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048765+0000", 2024-09-13T09:26:52.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048765+0000", 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048764+0000", 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048767+0000", 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048774+0000", 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.601e-06 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048778+0000", 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3220000000000003e-06 2024-09-13T09:26:52.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96701 3.1a 3:5bc525f1:::rbd_data.111838d2a860.0000000000000003:17 [read 26624~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052146+0000", 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0010061810000000001, 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0062689979999999996, 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96701 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052146+0000", 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052146+0000", 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052146+0000", 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052149+0000", 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.717e-06 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052154+0000", 2024-09-13T09:26:52.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052158+0000", 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9840000000000003e-06 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.057623+0000", 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054658349999999996 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.057689+0000", 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6019999999999995e-05 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96733 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052772+0000", 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00038065899999999999, 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056850379999999999, 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96733 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052772+0000", 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052772+0000", 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052772+0000", 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052774+0000", 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3530000000000002e-06 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052779+0000", 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5720000000000002e-06 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052784+0000", 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7439999999999998e-06 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.438 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96518 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.043759+0000", 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0093936799999999997, 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014730912000000001, 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96518 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043759+0000", 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043759+0000", 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043759+0000", 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043761+0000", 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043773+0000", 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1926000000000001e-05 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043778+0000", 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.181e-06 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.441 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96702 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052161+0000", 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00099121600000000006, 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0063827210000000001, 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96702 2024-09-13T09:26:52.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052161+0000", 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052161+0000", 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052161+0000", 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052163+0000", 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052169+0000", 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9789999999999998e-06 2024-09-13T09:26:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052176+0000", 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8340000000000001e-06 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96734 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052790+0000", 2024-09-13T09:26:52.445 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00036311900000000002, 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057893370000000003, 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96734 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052790+0000", 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052790+0000", 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052789+0000", 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052792+0000", 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5050000000000002e-06 2024-09-13T09:26:52.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052797+0000", 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.519e-06 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052802+0000", 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96204 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023550+0000", 2024-09-13T09:26:52.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029602594999999999, 2024-09-13T09:26:52.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035055248999999997, 2024-09-13T09:26:52.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96204 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023550+0000", 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023550+0000", 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023550+0000", 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023551+0000", 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5510000000000001e-06 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023554+0000", 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.892e-06 2024-09-13T09:26:52.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023559+0000", 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4050000000000002e-06 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96551 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.048801+0000", 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.004351588, 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0098469850000000008, 2024-09-13T09:26:52.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96551 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048801+0000", 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048801+0000", 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048801+0000", 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048803+0000", 2024-09-13T09:26:52.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.407e-06 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.048815+0000", 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1459e-05 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049010+0000", 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019541799999999999 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96703 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052196+0000", 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00095693299999999998, 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064998290000000004, 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96703 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052196+0000", 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052196+0000", 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052195+0000", 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052198+0000", 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9780000000000001e-06 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052204+0000", 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.516e-06 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052210+0000", 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1499999999999998e-06 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96735 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052806+0000", 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00034615600000000002, 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059168780000000004, 2024-09-13T09:26:52.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96735 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052806+0000", 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052806+0000", 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052806+0000", 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052808+0000", 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2259999999999999e-06 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052814+0000", 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3909999999999997e-06 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052819+0000", 2024-09-13T09:26:52.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1900000000000003e-06 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96205 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023572+0000", 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029580177999999999, 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035198291999999999, 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96205 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023572+0000", 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023572+0000", 2024-09-13T09:26:52.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023572+0000", 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023575+0000", 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023580+0000", 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023585+0000", 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96704 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.469 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052223+0000", 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00092939900000000002, 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006598652, 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96704 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052223+0000", 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052223+0000", 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052223+0000", 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052233+0000", 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0056e-05 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052239+0000", 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5050000000000004e-06 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052244+0000", 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:26:52.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96736 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052823+0000", 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00032988700000000001, 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060576830000000003, 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96736 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052823+0000", 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052823+0000", 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052822+0000", 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052825+0000", 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5179999999999999e-06 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052830+0000", 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3669999999999996e-06 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052835+0000", 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4649999999999996e-06 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96206 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.023593+0000", 2024-09-13T09:26:52.476 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.029559281999999999, 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035349037999999999, 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96206 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023593+0000", 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023593+0000", 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023593+0000", 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023596+0000", 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5969999999999999e-06 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023601+0000", 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6280000000000002e-06 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.023606+0000", 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7530000000000001e-06 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96521 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.043844+0000", 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0093083300000000001, 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015149848, 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96521 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043844+0000", 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043844+0000", 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043844+0000", 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043846+0000", 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3949999999999999e-06 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043852+0000", 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3340000000000001e-06 2024-09-13T09:26:52.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.043856+0000", 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96553 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.049038+0000", 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0041149130000000004, 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0099863609999999992, 2024-09-13T09:26:52.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96553 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049038+0000", 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049038+0000", 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049037+0000", 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049040+0000", 2024-09-13T09:26:52.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049045+0000", 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.049050+0000", 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6829999999999999e-06 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96705 3.1a 3.8fa4a3da (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052249+0000", 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00090391799999999997, 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068173799999999996, 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96705 2024-09-13T09:26:52.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052249+0000", 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052249+0000", 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052248+0000", 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052251+0000", 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6800000000000002e-06 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052257+0000", 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7239999999999997e-06 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052261+0000", 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5419999999999999e-06 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:96737 3.9 3.6b6a3cc9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.052839+0000", 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00031346899999999998, 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0062658009999999997, 2024-09-13T09:26:52.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "queued for pg", 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 96737 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052839+0000", 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052839+0000", 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052839+0000", 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052841+0000", 2024-09-13T09:26:52.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052847+0000", 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9710000000000001e-06 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.052852+0000", 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8160000000000001e-06 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:52.494 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:26:52.494 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 117 2024-09-13T09:26:52.494 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:52.494 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:52.495 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:52.487+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:52.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:52.596 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:52.592+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:52.643 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:26:52.650 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:52.657 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:52.657 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:52.657 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:52.667 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:52.670 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:52.689 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:52.696 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:52.687+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:52.698 INFO:teuthology.orchestra.run.smithi063.stdout:2526 write 0x13c1875 thru 0x13c6c75 (0x5401 bytes) 2024-09-13T09:26:52.712 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:52.729 INFO:teuthology.orchestra.run.smithi063.stdout:2527 write 0x8b0bdec thru 0x8b11c80 (0x5e95 bytes) 2024-09-13T09:26:52.738 INFO:teuthology.orchestra.run.smithi063.stdout:2528 write 0x327c6ed thru 0x3286681 (0x9f95 bytes) 2024-09-13T09:26:52.738 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:52.751 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:52.752 INFO:teuthology.orchestra.run.smithi063.stdout:2531 punch from 0x1a68d0b to 0x1a7270f, (0x9a04 bytes) 2024-09-13T09:26:52.755 INFO:teuthology.orchestra.run.smithi063.stdout:2534 trunc from 0x8b11c81 to 0x6c78f09 2024-09-13T09:26:52.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:52.771 INFO:teuthology.orchestra.run.smithi063.stdout:2535 read 0x47381fe thru 0x473fe2b (0x7c2e bytes) 2024-09-13T09:26:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:52.788 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:52.788 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:52.788 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:52.789 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:52.789 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:52.789 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:52.789 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:52.789 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:52.791 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:52.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:52.794 INFO:teuthology.orchestra.run.smithi063.stdout:2536 write 0x72448fe thru 0x724fad9 (0xb1dc bytes) 2024-09-13T09:26:52.797 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:52.791+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:52.802 INFO:teuthology.orchestra.run.smithi063.stdout:2537 read 0x589da73 thru 0x589fad2 (0x2060 bytes) 2024-09-13T09:26:52.803 INFO:teuthology.orchestra.run.smithi063.stdout:2538 punch from 0x1094be5 to 0x109567f, (0xa9a bytes) 2024-09-13T09:26:52.803 INFO:teuthology.orchestra.run.smithi063.stdout:2543 punch from 0x487f618 to 0x48821ae, (0x2b96 bytes) 2024-09-13T09:26:52.823 INFO:teuthology.orchestra.run.smithi063.stdout:2546 write 0xcde724b thru 0xcdf3732 (0xc4e8 bytes) 2024-09-13T09:26:52.831 INFO:teuthology.orchestra.run.smithi063.stdout:2547 writesame 0x5a9aeec thru 0x5aa4c09 data_size 0x77(0x9d1e bytes) 2024-09-13T09:26:52.834 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:52.834 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:52.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:52.846 INFO:teuthology.orchestra.run.smithi063.stdout:2548 read 0x21cd092 thru 0x21d91b6 (0xc125 bytes) 2024-09-13T09:26:52.848 INFO:teuthology.orchestra.run.smithi063.stdout:2549 write 0x1ba97db thru 0x1bb7965 (0xe18b bytes) 2024-09-13T09:26:52.866 INFO:teuthology.orchestra.run.smithi063.stdout:2550 read 0x9a479b1 thru 0x9a48094 (0x6e4 bytes) 2024-09-13T09:26:52.866 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:52.866 INFO:teuthology.orchestra.run.smithi063.stdout:2552 read 0x2f261dd thru 0x2f306f1 (0xa515 bytes) 2024-09-13T09:26:52.867 INFO:teuthology.orchestra.run.smithi063.stdout:2553 read 0xcb0a864 thru 0xcb0eefc (0x4699 bytes) 2024-09-13T09:26:52.888 INFO:teuthology.orchestra.run.smithi063.stdout:2557 trunc from 0xcdf3733 to 0x37f5b6f 2024-09-13T09:26:52.897 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:52.896+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:52.920 INFO:teuthology.orchestra.run.smithi063.stdout:2559 punch from 0x2e865f6 to 0x2e8d45f, (0x6e69 bytes) 2024-09-13T09:26:52.920 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:52.941 INFO:teuthology.orchestra.run.smithi063.stdout:2561 trunc from 0x37f5b6f to 0xd466026 2024-09-13T09:26:52.949 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:52.949 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:52.954 INFO:teuthology.orchestra.run.smithi063.stdout:2562 write 0x929e803 thru 0x92a4c8c (0x648a bytes) 2024-09-13T09:26:52.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:52.963 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:52.963 INFO:teuthology.orchestra.run.smithi063.stdout:2568 punch from 0xd2e571b to 0xd2f397d, (0xe262 bytes) 2024-09-13T09:26:52.963 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:52.963 INFO:teuthology.orchestra.run.smithi063.stdout:2572 flatten 2024-09-13T09:26:53.001 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:53.000+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:53.073 INFO:teuthology.orchestra.run.smithi063.stdout:2575 read 0xa54f58f thru 0xa555b55 (0x65c7 bytes) 2024-09-13T09:26:53.073 INFO:teuthology.orchestra.run.smithi063.stdout:2578 read 0x155ab2c thru 0x15606cb (0x5ba0 bytes) 2024-09-13T09:26:53.073 INFO:teuthology.orchestra.run.smithi063.stdout:2579 read 0x7090154 thru 0x7099f00 (0x9dad bytes) 2024-09-13T09:26:53.073 INFO:teuthology.orchestra.run.smithi063.stdout:2580 read 0x125f44c thru 0x126ea05 (0xf5ba bytes) 2024-09-13T09:26:53.074 INFO:teuthology.orchestra.run.smithi063.stdout:2581 trunc from 0xd466026 to 0xe02755d 2024-09-13T09:26:53.084 INFO:teuthology.orchestra.run.smithi063.stdout:2582 read 0x287f57f thru 0x2880bec (0x166e bytes) 2024-09-13T09:26:53.084 INFO:teuthology.orchestra.run.smithi063.stdout:2583 punch from 0x2942ae0 to 0x294facc, (0xcfec bytes) 2024-09-13T09:26:53.094 INFO:teuthology.orchestra.run.smithi063.stdout:2584 write 0xe0f6f07 thru 0xe0fe74e (0x7848 bytes) 2024-09-13T09:26:53.100 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:53.091+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:2586 read 0x518d75e thru 0x5191009 (0x38ac bytes) 2024-09-13T09:26:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:2588 read 0x7e78f92 thru 0x7e827e9 (0x9858 bytes) 2024-09-13T09:26:53.101 INFO:teuthology.orchestra.run.smithi063.stdout:2589 write 0x23ce761 thru 0x23da3a6 (0xbc46 bytes) 2024-09-13T09:26:53.110 INFO:teuthology.orchestra.run.smithi063.stdout:2591 write 0x516d979 thru 0x5170aa5 (0x312d bytes) 2024-09-13T09:26:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.119 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.119 INFO:teuthology.orchestra.run.smithi063.stdout:2592 writesame 0xb85ccc5 thru 0xb865707 data_size 0x1(0x8a43 bytes) 2024-09-13T09:26:53.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:53.144 INFO:teuthology.orchestra.run.smithi063.stdout:2593 read 0xc442fdf thru 0xc44d532 (0xa554 bytes) 2024-09-13T09:26:53.144 INFO:teuthology.orchestra.run.smithi063.stdout:2594 read 0xd9b9346 thru 0xd9b9ebf (0xb7a bytes) 2024-09-13T09:26:53.144 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.144 INFO:teuthology.orchestra.run.smithi063.stdout:2597 write 0xfdc26c thru 0xfdfd49 (0x3ade bytes) 2024-09-13T09:26:53.146 INFO:teuthology.orchestra.run.smithi063.stdout:2598 write 0x4079e6a thru 0x407c986 (0x2b1d bytes) 2024-09-13T09:26:53.151 INFO:teuthology.orchestra.run.smithi063.stdout:2599 read 0x70a87f2 thru 0x70b4f68 (0xc777 bytes) 2024-09-13T09:26:53.151 INFO:teuthology.orchestra.run.smithi063.stdout:2601 read 0x17e747a thru 0x17f4340 (0xcec7 bytes) 2024-09-13T09:26:53.151 INFO:teuthology.orchestra.run.smithi063.stdout:2602 write 0x2ee7344 thru 0x2ef3889 (0xc546 bytes) 2024-09-13T09:26:53.161 INFO:teuthology.orchestra.run.smithi063.stdout:2603 writesame 0x13eaa0a thru 0x13f3dec data_size 0x11(0x93e3 bytes) 2024-09-13T09:26:53.182 INFO:teuthology.orchestra.run.smithi063.stdout:2604 writesame 0xea6d068 thru 0xea751c1 data_size 0x3(0x815a bytes) 2024-09-13T09:26:53.182 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.182 INFO:teuthology.orchestra.run.smithi063.stdout:2606 write 0x4920936 thru 0x493032c (0xf9f7 bytes) 2024-09-13T09:26:53.190 INFO:teuthology.orchestra.run.smithi063.stdout:2608 read 0x2fe6669 thru 0x2ff051e (0x9eb6 bytes) 2024-09-13T09:26:53.192 INFO:teuthology.orchestra.run.smithi063.stdout:2611 read 0xb3b2892 thru 0xb3b3531 (0xca0 bytes) 2024-09-13T09:26:53.192 INFO:teuthology.orchestra.run.smithi063.stdout:2612 read 0x2b10b35 thru 0x2b16fd6 (0x64a2 bytes) 2024-09-13T09:26:53.192 INFO:teuthology.orchestra.run.smithi063.stdout:2613 read 0xd113175 thru 0xd11edb1 (0xbc3d bytes) 2024-09-13T09:26:53.192 INFO:teuthology.orchestra.run.smithi063.stdout:2615 writesame 0x59db451 thru 0x59e52a4 data_size 0x2(0x9e54 bytes) 2024-09-13T09:26:53.200 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:53.196+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:53.210 INFO:teuthology.orchestra.run.smithi063.stdout:2617 read 0x4c97588 thru 0x4ca3eb9 (0xc932 bytes) 2024-09-13T09:26:53.212 INFO:teuthology.orchestra.run.smithi063.stdout:2618 read 0xe80ab68 thru 0xe817bc4 (0xd05d bytes) 2024-09-13T09:26:53.212 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.212 INFO:teuthology.orchestra.run.smithi063.stdout:2622 punch from 0x3116450 to 0x312170a, (0xb2ba bytes) 2024-09-13T09:26:53.212 INFO:teuthology.orchestra.run.smithi063.stdout:2624 read 0x999fb53 thru 0x99aee5f (0xf30d bytes) 2024-09-13T09:26:53.212 INFO:teuthology.orchestra.run.smithi063.stdout:2625 writesame 0xaefefbe thru 0xaf072c9 data_size 0x2(0x830c bytes) 2024-09-13T09:26:53.224 INFO:teuthology.orchestra.run.smithi063.stdout:2626 read 0x745d103 thru 0x7462758 (0x5656 bytes) 2024-09-13T09:26:53.224 INFO:teuthology.orchestra.run.smithi063.stdout:2627 read 0x897b28c thru 0x897c65c (0x13d1 bytes) 2024-09-13T09:26:53.228 INFO:teuthology.orchestra.run.smithi063.stdout:2628 trunc from 0xea751c2 to 0xd6bb57a 2024-09-13T09:26:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.242 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.247 INFO:teuthology.orchestra.run.smithi063.stdout:2632 read 0x3a80f82 thru 0x3a8a947 (0x99c6 bytes) 2024-09-13T09:26:53.248 INFO:teuthology.orchestra.run.smithi063.stdout:2634 writesame 0xcd2b0aa thru 0xcd391f0 data_size 0x1(0xe147 bytes) 2024-09-13T09:26:53.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:53.301 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:53.300+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:53.301 INFO:teuthology.orchestra.run.smithi063.stdout:2635 trunc from 0xd6bb57a to 0x29a1508 2024-09-13T09:26:53.329 INFO:teuthology.orchestra.run.smithi063.stdout:2636 read 0xcba70e thru 0xcbf853 (0x5146 bytes) 2024-09-13T09:26:53.361 INFO:teuthology.orchestra.run.smithi063.stdout:2638 write 0xdcdce21 thru 0xdce2707 (0x58e7 bytes) 2024-09-13T09:26:53.368 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.368 INFO:teuthology.orchestra.run.smithi063.stdout:2641 writesame 0xa0e8fb1 thru 0xa0f3777 data_size 0x3(0xa7c7 bytes) 2024-09-13T09:26:53.385 INFO:teuthology.orchestra.run.smithi063.stdout:2642 write 0x9bf0db4 thru 0x9bfbb2b (0xad78 bytes) 2024-09-13T09:26:53.394 INFO:teuthology.orchestra.run.smithi063.stdout:2643 write 0xc7629ef thru 0xc76b5d2 (0x8be4 bytes) 2024-09-13T09:26:53.402 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:53.400+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:53.408 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.408 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.418 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:2645 trunc from 0xdce2708 to 0x6b72ebe 2024-09-13T09:26:53.443 INFO:teuthology.orchestra.run.smithi063.stdout:2646 trunc from 0x6b72ebe to 0x5c19150 2024-09-13T09:26:53.455 INFO:teuthology.orchestra.run.smithi063.stdout:2647 read 0x1a1a979 thru 0x1a1d7bd (0x2e45 bytes) 2024-09-13T09:26:53.455 INFO:teuthology.orchestra.run.smithi063.stdout:2649 writesame 0x1e7743 thru 0x1eb870 data_size 0x51(0x412e bytes) 2024-09-13T09:26:53.460 INFO:teuthology.orchestra.run.smithi063.stdout:2650 read 0x58b86d7 thru 0x58c0016 (0x7940 bytes) 2024-09-13T09:26:53.489 INFO:teuthology.orchestra.run.smithi063.stdout:2652 writesame 0xe365b80 thru 0xe36c7d8 data_size 0x1(0x6c59 bytes) 2024-09-13T09:26:53.490 INFO:teuthology.orchestra.run.smithi063.stdout:2655 writesame 0x68488df thru 0x684c6b4 data_size 0x5(0x3dd6 bytes) 2024-09-13T09:26:53.498 INFO:teuthology.orchestra.run.smithi063.stdout:2656 read 0x73e50b5 thru 0x73e77b8 (0x2704 bytes) 2024-09-13T09:26:53.498 INFO:teuthology.orchestra.run.smithi063.stdout:2657 write 0x7230a26 thru 0x723627c (0x5857 bytes) 2024-09-13T09:26:53.502 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:53.500+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:53.504 INFO:teuthology.orchestra.run.smithi063.stdout:2658 write 0x7b8dbd3 thru 0x7b9954c (0xb97a bytes) 2024-09-13T09:26:53.514 INFO:teuthology.orchestra.run.smithi063.stdout:2661 write 0x3d0e21 thru 0x3d4c08 (0x3de8 bytes) 2024-09-13T09:26:53.519 INFO:teuthology.orchestra.run.smithi063.stdout:2662 writesame 0x4a8d25f thru 0x4a95948 data_size 0x7(0x86ea bytes) 2024-09-13T09:26:53.524 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.525 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.528 INFO:teuthology.orchestra.run.smithi063.stdout:2663 punch from 0x4943a63 to 0x4948237, (0x47d4 bytes) 2024-09-13T09:26:53.528 INFO:teuthology.orchestra.run.smithi063.stdout:2665 write 0x2d14dbb thru 0x2d14dfc (0x42 bytes) 2024-09-13T09:26:53.534 INFO:teuthology.orchestra.run.smithi063.stdout:2666 punch from 0x62d0e52 to 0x62d1346, (0x4f4 bytes) 2024-09-13T09:26:53.534 INFO:teuthology.orchestra.run.smithi063.stdout:2667 write 0x46ea92b thru 0x46f486b (0x9f41 bytes) 2024-09-13T09:26:53.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:2672 punch from 0xe7b846 to 0xe8462a, (0x8de4 bytes) 2024-09-13T09:26:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:2674 read 0xa1ee65d thru 0xa1fc0e7 (0xda8b bytes) 2024-09-13T09:26:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:2676 read 0xbddef44 thru 0xbde43f0 (0x54ad bytes) 2024-09-13T09:26:53.552 INFO:teuthology.orchestra.run.smithi063.stdout:2677 trunc from 0xe36c7d9 to 0x9d6e2d2 2024-09-13T09:26:53.572 INFO:teuthology.orchestra.run.smithi063.stdout:2681 trunc from 0x9d6e2d2 to 0xe3cea22 2024-09-13T09:26:53.583 INFO:teuthology.orchestra.run.smithi063.stdout:2682 read 0x26a4702 thru 0x26a78e6 (0x31e5 bytes) 2024-09-13T09:26:53.584 INFO:teuthology.orchestra.run.smithi063.stdout:2683 write 0xd8a2f61 thru 0xd8a4263 (0x1303 bytes) 2024-09-13T09:26:53.590 INFO:teuthology.orchestra.run.smithi063.stdout:2684 punch from 0xc30e34e to 0xc319e26, (0xbad8 bytes) 2024-09-13T09:26:53.591 INFO:teuthology.orchestra.run.smithi063.stdout:2685 read 0x86d61de thru 0x86d8152 (0x1f75 bytes) 2024-09-13T09:26:53.594 INFO:teuthology.orchestra.run.smithi063.stdout:2686 trunc from 0xe3cea22 to 0xd07a1b5 2024-09-13T09:26:53.603 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:53.600+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:53.614 INFO:teuthology.orchestra.run.smithi063.stdout:2687 punch from 0x85e5a10 to 0x85ed0db, (0x76cb bytes) 2024-09-13T09:26:53.614 INFO:teuthology.orchestra.run.smithi063.stdout:2688 read 0x86b802a thru 0x86c6294 (0xe26b bytes) 2024-09-13T09:26:53.614 INFO:teuthology.orchestra.run.smithi063.stdout:2690 read 0x1cbe482 thru 0x1cc8b6a (0xa6e9 bytes) 2024-09-13T09:26:53.614 INFO:teuthology.orchestra.run.smithi063.stdout:2691 read 0x7908a2c thru 0x791101d (0x85f2 bytes) 2024-09-13T09:26:53.614 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0xee0ba01 2024-09-13T09:26:53.619 INFO:teuthology.orchestra.run.smithi063.stdout:2692 trunc from 0xd07a1b5 to 0xee0ba01 2024-09-13T09:26:53.631 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.631 INFO:teuthology.orchestra.run.smithi063.stdout:2695 write 0xe3437be thru 0xe34ef9a (0xb7dd bytes) 2024-09-13T09:26:53.641 INFO:teuthology.orchestra.run.smithi063.stdout:2697 punch from 0x5d46bc9 to 0x5d4d813, (0x6c4a bytes) 2024-09-13T09:26:53.646 INFO:teuthology.orchestra.run.smithi063.stdout:2699 trunc from 0xee0ba01 to 0xc98fc4a 2024-09-13T09:26:53.666 INFO:teuthology.orchestra.run.smithi063.stdout:2700 write 0xb002390 thru 0xb00edb1 (0xca22 bytes) 2024-09-13T09:26:53.693 INFO:teuthology.orchestra.run.smithi063.stdout:2702 write 0xd83a673 thru 0xd8445c8 (0x9f56 bytes) 2024-09-13T09:26:53.693 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.693 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.698 INFO:teuthology.orchestra.run.smithi063.stdout:2703 read 0xaf1cd5 thru 0xaf5924 (0x3c50 bytes) 2024-09-13T09:26:53.698 INFO:teuthology.orchestra.run.smithi063.stdout:2705 writesame 0xc4c6785 thru 0xc4d17fd data_size 0x25(0xb079 bytes) 2024-09-13T09:26:53.704 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:53.695+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:53.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:53.709 INFO:teuthology.orchestra.run.smithi063.stdout:2706 write 0x5f5082e thru 0x5f5650a (0x5cdd bytes) 2024-09-13T09:26:53.717 INFO:teuthology.orchestra.run.smithi063.stdout:2707 trunc from 0xd8445c9 to 0xcd96dcb 2024-09-13T09:26:53.738 INFO:teuthology.orchestra.run.smithi063.stdout:2708 punch from 0x8e6200f to 0x8e63c9d, (0x1c8e bytes) 2024-09-13T09:26:53.738 INFO:teuthology.orchestra.run.smithi063.stdout:2710 write 0x4f97d2c thru 0x4f9df24 (0x61f9 bytes) 2024-09-13T09:26:53.747 INFO:teuthology.orchestra.run.smithi063.stdout:2711 writesame 0x3c19f85 thru 0x3c1afcf data_size 0x1(0x104b bytes) 2024-09-13T09:26:53.756 INFO:teuthology.orchestra.run.smithi063.stdout:2712 read 0xc39221b thru 0xc398da2 (0x6b88 bytes) 2024-09-13T09:26:53.759 INFO:teuthology.orchestra.run.smithi063.stdout:2713 trunc from 0xcd96dcb to 0xe0bcefd 2024-09-13T09:26:53.794 INFO:teuthology.orchestra.run.smithi063.stdout:2714 trunc from 0xe0bcefd to 0x40900ef 2024-09-13T09:26:53.805 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:53.795+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:53.814 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.814 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:53.849 INFO:teuthology.orchestra.run.smithi063.stdout:2716 writesame 0xd94616c thru 0xd950a60 data_size 0x25(0xa8f5 bytes) 2024-09-13T09:26:53.849 INFO:teuthology.orchestra.run.smithi063.stdout:2717 read 0x232202c thru 0x232fd65 (0xdd3a bytes) 2024-09-13T09:26:53.849 INFO:teuthology.orchestra.run.smithi063.stdout:2718 write 0xa9466c0 thru 0xa9516fa (0xb03b bytes) 2024-09-13T09:26:53.858 INFO:teuthology.orchestra.run.smithi063.stdout:2719 writesame 0xc90190a thru 0xc9118d9 data_size 0x8(0xffd0 bytes) 2024-09-13T09:26:53.872 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.874 INFO:teuthology.orchestra.run.smithi063.stdout:2723 trunc from 0xd950a61 to 0xe7b1edc 2024-09-13T09:26:53.885 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:53.885 INFO:teuthology.orchestra.run.smithi063.stdout:2725 punch from 0xbb6c490 to 0xbb717ff, (0x536f bytes) 2024-09-13T09:26:53.885 INFO:teuthology.orchestra.run.smithi063.stdout:2727 write 0x6a68c3d thru 0x6a725f9 (0x99bd bytes) 2024-09-13T09:26:53.897 INFO:teuthology.orchestra.run.smithi063.stdout:2729 punch from 0xd6e6033 to 0xd6e6e42, (0xe0f bytes) 2024-09-13T09:26:53.897 INFO:teuthology.orchestra.run.smithi063.stdout:2731 punch from 0x7633f27 to 0x7637016, (0x30ef bytes) 2024-09-13T09:26:53.897 INFO:teuthology.orchestra.run.smithi063.stdout:2732 punch from 0xb81ba54 to 0xb8207c9, (0x4d75 bytes) 2024-09-13T09:26:53.905 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:53.904+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:53.911 INFO:teuthology.orchestra.run.smithi063.stdout:2733 trunc from 0xe7b1edc to 0x6db93f1 2024-09-13T09:26:53.925 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:53.926 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:53.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:53.944 INFO:teuthology.orchestra.run.smithi063.stdout:2734 trunc from 0x6db93f1 to 0x26a7f09 2024-09-13T09:26:53.966 INFO:teuthology.orchestra.run.smithi063.stdout:2735 punch from 0x1ebc817 to 0x1ecb9a4, (0xf18d bytes) 2024-09-13T09:26:53.966 INFO:teuthology.orchestra.run.smithi063.stdout:2736 read 0x23495c3 thru 0x234ebe8 (0x5626 bytes) 2024-09-13T09:26:53.971 INFO:teuthology.orchestra.run.smithi063.stdout:2738 trunc from 0x26a7f09 to 0x492dcca 2024-09-13T09:26:53.983 INFO:teuthology.orchestra.run.smithi063.stdout:2740 read 0x2752b5 thru 0x281de5 (0xcb31 bytes) 2024-09-13T09:26:54.001 INFO:teuthology.orchestra.run.smithi063.stdout:2741 trunc from 0x492dcca to 0xc6c95dd 2024-09-13T09:26:54.006 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:54.004+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:54.013 INFO:teuthology.orchestra.run.smithi063.stdout:2742 write 0x38dbe28 thru 0x38eaf3f (0xf118 bytes) 2024-09-13T09:26:54.024 INFO:teuthology.orchestra.run.smithi063.stdout:2743 write 0xa31d371 thru 0xa31d95e (0x5ee bytes) 2024-09-13T09:26:54.029 INFO:teuthology.orchestra.run.smithi063.stdout:2745 write 0x6dda48a thru 0x6de6f8f (0xcb06 bytes) 2024-09-13T09:26:54.035 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.038 INFO:teuthology.orchestra.run.smithi063.stdout:2747 trunc from 0xc6c95dd to 0xdaebd04 2024-09-13T09:26:54.040 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.041 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.050 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:54.051 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.051 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.051 INFO:teuthology.orchestra.run.smithi063.stdout:2751 read 0x743bd16 thru 0x744a725 (0xea10 bytes) 2024-09-13T09:26:54.057 INFO:teuthology.orchestra.run.smithi063.stdout:2752 trunc from 0xdaebd04 to 0xb42e975 2024-09-13T09:26:54.067 INFO:teuthology.orchestra.run.smithi063.stdout:2753 write 0x891001c thru 0x891267a (0x265f bytes) 2024-09-13T09:26:54.092 INFO:teuthology.orchestra.run.smithi063.stdout:2754 write 0xedcfc90 thru 0xedd066c (0x9dd bytes) 2024-09-13T09:26:54.101 INFO:teuthology.orchestra.run.smithi063.stdout:2755 read 0xd22ce4 thru 0xd31e00 (0xf11d bytes) 2024-09-13T09:26:54.102 INFO:teuthology.orchestra.run.smithi063.stdout:2756 writesame 0xc1be2a1 thru 0xc1c627e data_size 0xd(0x7fde bytes) 2024-09-13T09:26:54.107 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:54.104+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:54.111 INFO:teuthology.orchestra.run.smithi063.stdout:2757 read 0x65c9445 thru 0x65d1ed1 (0x8a8d bytes) 2024-09-13T09:26:54.136 INFO:teuthology.orchestra.run.smithi063.stdout:2758 trunc from 0xedd066d to 0x6db9d2 2024-09-13T09:26:54.166 INFO:teuthology.orchestra.run.smithi063.stdout:2759 read 0x4e5b19 thru 0x4f1c50 (0xc138 bytes) 2024-09-13T09:26:54.167 INFO:teuthology.orchestra.run.smithi063.stdout:2760 read 0x4f4dc2 thru 0x501492 (0xc6d1 bytes) 2024-09-13T09:26:54.168 INFO:teuthology.orchestra.run.smithi063.stdout:2761 punch from 0x55ac16 to 0x567c8d, (0xd077 bytes) 2024-09-13T09:26:54.169 INFO:teuthology.orchestra.run.smithi063.stdout:2763 read 0x352180 thru 0x3567fd (0x467e bytes) 2024-09-13T09:26:54.175 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.175 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.189 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:54.198 INFO:teuthology.orchestra.run.smithi063.stdout:2764 write 0xcc18068 thru 0xcc1b12c (0x30c5 bytes) 2024-09-13T09:26:54.204 INFO:teuthology.orchestra.run.smithi063.stdout:2766 punch from 0x6960482 to 0x69647e1, (0x435f bytes) 2024-09-13T09:26:54.204 INFO:teuthology.orchestra.run.smithi063.stdout:2767 write 0x741b1be thru 0x742b08e (0xfed1 bytes) 2024-09-13T09:26:54.207 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:54.204+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:54.216 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.216 INFO:teuthology.orchestra.run.smithi063.stdout:2770 write 0x5fe0d1a thru 0x5fe92ab (0x8592 bytes) 2024-09-13T09:26:54.223 INFO:teuthology.orchestra.run.smithi063.stdout:2771 writesame 0x7310f26 thru 0x7319ab7 data_size 0x12(0x8b92 bytes) 2024-09-13T09:26:54.230 INFO:teuthology.orchestra.run.smithi063.stdout:2772 punch from 0xa81e69f to 0xa81ebe5, (0x546 bytes) 2024-09-13T09:26:54.230 INFO:teuthology.orchestra.run.smithi063.stdout:2773 write 0xa5825b4 thru 0xa58665c (0x40a9 bytes) 2024-09-13T09:26:54.235 INFO:teuthology.orchestra.run.smithi063.stdout:2774 write 0x1ffe139 thru 0x200bfcc (0xde94 bytes) 2024-09-13T09:26:54.259 INFO:teuthology.orchestra.run.smithi063.stdout:2775 write 0xe318ccc thru 0xe321ffc (0x9331 bytes) 2024-09-13T09:26:54.266 INFO:teuthology.orchestra.run.smithi063.stdout:2776 punch from 0x142dcb to 0x151f6e, (0xf1a3 bytes) 2024-09-13T09:26:54.266 INFO:teuthology.orchestra.run.smithi063.stdout:2777 write 0xcf3e082 thru 0xcf4825d (0xa1dc bytes) 2024-09-13T09:26:54.276 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.276 INFO:teuthology.orchestra.run.smithi063.stdout:2779 read 0x9e2be8f thru 0x9e2e937 (0x2aa9 bytes) 2024-09-13T09:26:54.276 INFO:teuthology.orchestra.run.smithi063.stdout:2780 read 0xcc0478a thru 0xcc11c27 (0xd49e bytes) 2024-09-13T09:26:54.296 INFO:teuthology.orchestra.run.smithi063.stdout:2782 trunc from 0xe321ffd to 0x2398262 2024-09-13T09:26:54.308 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:54.304+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:54.314 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.314 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.317 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.317 INFO:teuthology.orchestra.run.smithi063.stdout:2785 punch from 0xee4214 to 0xee50a9, (0xe95 bytes) 2024-09-13T09:26:54.317 INFO:teuthology.orchestra.run.smithi063.stdout:2787 read 0x78e964 thru 0x7980d8 (0x9775 bytes) 2024-09-13T09:26:54.327 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:54.339 INFO:teuthology.orchestra.run.smithi063.stdout:2788 writesame 0x9093a0d thru 0x9094535 data_size 0x1(0xb29 bytes) 2024-09-13T09:26:54.339 INFO:teuthology.orchestra.run.smithi063.stdout:2790 punch from 0x4af0fb7 to 0x4af8e98, (0x7ee1 bytes) 2024-09-13T09:26:54.351 INFO:teuthology.orchestra.run.smithi063.stdout:2791 write 0x9a8149a thru 0x9a8ef32 (0xda99 bytes) 2024-09-13T09:26:54.359 INFO:teuthology.orchestra.run.smithi063.stdout:2793 punch from 0x99ea3ad to 0x99ebf3f, (0x1b92 bytes) 2024-09-13T09:26:54.371 INFO:teuthology.orchestra.run.smithi063.stdout:2794 write 0x9c947eb thru 0x9c9ce76 (0x868c bytes) 2024-09-13T09:26:54.378 INFO:teuthology.orchestra.run.smithi063.stdout:2796 punch from 0x7583681 to 0x7592a14, (0xf393 bytes) 2024-09-13T09:26:54.378 INFO:teuthology.orchestra.run.smithi063.stdout:2797 write 0x357a71a thru 0x3587e14 (0xd6fb bytes) 2024-09-13T09:26:54.409 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:54.403+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:54.446 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.446 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.456 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:54.510 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:54.508+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:54.578 INFO:teuthology.orchestra.run.smithi063.stdout:2800 write 0x2daaf10 thru 0x2db82e8 (0xd3d9 bytes) 2024-09-13T09:26:54.591 INFO:teuthology.orchestra.run.smithi063.stdout:2801 read 0x39ead3c thru 0x39fa868 (0xfb2d bytes) 2024-09-13T09:26:54.591 INFO:teuthology.orchestra.run.smithi063.stdout:2802 punch from 0x2eb5a4 to 0x2ef3df, (0x3e3b bytes) 2024-09-13T09:26:54.593 INFO:teuthology.orchestra.run.smithi063.stdout:2803 trunc from 0x9c9ce77 to 0x8b1761b 2024-09-13T09:26:54.610 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.610 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.610 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:54.608+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:54.612 INFO:teuthology.orchestra.run.smithi063.stdout:2804 read 0x6a8bda3 thru 0x6a9935b (0xd5b9 bytes) 2024-09-13T09:26:54.612 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.612 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.612 INFO:teuthology.orchestra.run.smithi063.stdout:2807 write 0x323efb9 thru 0x324eb57 (0xfb9f bytes) 2024-09-13T09:26:54.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:54.622 INFO:teuthology.orchestra.run.smithi063.stdout:2809 punch from 0x5540de6 to 0x554d28d, (0xc4a7 bytes) 2024-09-13T09:26:54.622 INFO:teuthology.orchestra.run.smithi063.stdout:2811 read 0x126aa0e thru 0x1275182 (0xa775 bytes) 2024-09-13T09:26:54.625 INFO:teuthology.orchestra.run.smithi063.stdout:2812 trunc from 0x8b1761b to 0x6f9f155 2024-09-13T09:26:54.641 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.653 INFO:teuthology.orchestra.run.smithi063.stdout:2814 trunc from 0x6f9f155 to 0xed073e2 2024-09-13T09:26:54.657 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:26:54.664 INFO:teuthology.orchestra.run.smithi063.stdout:2816 read 0xd0ce6f6 thru 0xd0de4cd (0xfdd8 bytes) 2024-09-13T09:26:54.664 INFO:teuthology.orchestra.run.smithi063.stdout:2817 write 0xb93d914 thru 0xb947aee (0xa1db bytes) 2024-09-13T09:26:54.673 INFO:teuthology.orchestra.run.smithi063.stdout:2818 punch from 0x14d254a to 0x14d5ea8, (0x395e bytes) 2024-09-13T09:26:54.673 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:54.674 INFO:teuthology.orchestra.run.smithi063.stdout:2822 punch from 0x92280f7 to 0x9237a17, (0xf920 bytes) 2024-09-13T09:26:54.675 INFO:teuthology.orchestra.run.smithi063.stdout:2823 trunc from 0xed073e2 to 0xdd55bbd 2024-09-13T09:26:54.684 INFO:teuthology.orchestra.run.smithi063.stdout:2824 clone 23 order 25 su 262144 sc 13 2024-09-13T09:26:54.711 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:54.708+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:54.724 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.724 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.734 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:54.793 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:54.812 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:54.808+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:54.877 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:54.877 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:54.887 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:54.913 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:54.912+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:54.934 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:54.956 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:54.982 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:54.998 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:54.998 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:54.998 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:26:54.998 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:55.003 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:55.008 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:55.014 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:55.012+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:55.027 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:55.051 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:55.074 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:55.075 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:55.079 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:55.079 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:55.109 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:55.110 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:26:55.110 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:26:55.110 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:26:55.110 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:55.114 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:55.112+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:55.120 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:55.215 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:55.207+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:55.273 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:26:55.273 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:26:55.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:26:55.273 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:26:55.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:276 3.4 e27/21 3:27927849:::rbd_data.111867511a81.0000000000000026:head v 27'1, mlcod=0'0)", 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.763731518, 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002929961, 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504326+0000", 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504324+0000", 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504366+0000", 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2412e-05 2024-09-13T09:26:55.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504407+0000", 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0329e-05 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504427+0000", 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0089999999999999e-05 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504518+0000", 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0801e-05 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.504566+0000", 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8745999999999997e-05 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507192+0000", 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00262546 2024-09-13T09:26:55.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.507256+0000", 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3769000000000006e-05 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.465118523000001, 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:26:55.278 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:26:55.282 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.056942638000002, 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:26:55.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.056752514999999, 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:26:55.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:26:55.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.055940106000001, 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:55.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 49.867033456999998, 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.295 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:26:55.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.301924071999998, 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:26:55.300 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:26:55.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.304 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.228385058999997, 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:26:55.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:26:55.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.135841691000003, 2024-09-13T09:26:55.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.107781115000002, 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:26:55.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.093523332, 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:26:55.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.324 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:26:55.324 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:55.324 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.624804878999999, 2024-09-13T09:26:55.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:26:55.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:26:55.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:26:55.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.635880186000001, 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:55.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:26:55.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:55.333 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.430854138000001, 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:26:55.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:26:55.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.332516089999999, 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:26:55.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.331707484999999, 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:26:55.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:26:55.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:26:55.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.2571149150000007, 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:26:55.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0630324719999997, 2024-09-13T09:26:55.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:26:55.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.355 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.9474109730000002, 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.678669631, 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:26:55.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:26:55.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:26:55.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.364 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:26:55.365 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:26:55.365 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:26:55.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:55.367 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:55.359+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:55.467 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:55.463+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:55.483 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:55.483 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:55.483 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:55.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:55.568 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:55.564+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:55.598 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:55.598 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:55.598 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:55.598 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:55.598 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:55.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:55.640 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-09-13T09:26:55.640 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:26:55.640 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-13T09:26:55.641 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T09:26:55.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T09:26:55.643 INFO:tasks.ceph.osd.3:Started 2024-09-13T09:26:55.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:26:55.658 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:26:55.669 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:55.663+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:55.719 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0229743629999994, 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:26:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:26:55.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.724 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.022951849, 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:26:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:26:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0229360120000006, 2024-09-13T09:26:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:26:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:26:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:26:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0227390679999999, 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.022714744, 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:26:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0226982840000005, 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0226822630000001, 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:26:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0226319299999993, 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:26:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0225283550000004, 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:26:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0131505589999996, 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:26:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:26:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0131318950000008, 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:26:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0131154050000006, 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0130996200000002, 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:26:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:26:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0130834990000004, 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:26:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0130675940000007, 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:26:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0130511250000005, 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:26:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0130292730000008, 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:26:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:26:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0130124580000004, 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0129961610000002, 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:26:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:26:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0129805600000008, 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:26:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:26:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:55.814 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:55.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:no valid command found; 10 closest matches: 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:0 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:1 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:2 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:abort 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:assert 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:config diff 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:config diff get 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:config get 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:config help [] 2024-09-13T09:26:55.817 INFO:teuthology.orchestra.run.smithi063.stderr:config set ... 2024-09-13T09:26:55.818 INFO:teuthology.orchestra.run.smithi063.stderr:admin_socket: invalid command 2024-09-13T09:26:55.818 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:55.818 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-13T09:26:55.818 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:55.807+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:55.917 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:55.911+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:55.932 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T09:26:55.937 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:55.937 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:55.937 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:55.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:55.946 INFO:teuthology.orchestra.run.smithi063.stdout:2825 read 0xca9848e thru 0xca9cb23 (0x4696 bytes) 2024-09-13T09:26:55.959 INFO:teuthology.orchestra.run.smithi063.stdout:2826 write 0x432079f thru 0x4328098 (0x78fa bytes) 2024-09-13T09:26:55.970 INFO:teuthology.orchestra.run.smithi063.stdout:2827 read 0x1d4c2ac thru 0x1d58043 (0xbd98 bytes) 2024-09-13T09:26:55.973 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:55.973 INFO:teuthology.orchestra.run.smithi063.stdout:2829 write 0x58cee2c thru 0x58de15b (0xf330 bytes) 2024-09-13T09:26:55.976 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:55.977 INFO:teuthology.orchestra.run.smithi063.stdout:2833 read 0x821ef0d thru 0x822965b (0xa74f bytes) 2024-09-13T09:26:55.977 INFO:teuthology.orchestra.run.smithi063.stdout:2834 write 0x5a6222c thru 0x5a6f25e (0xd033 bytes) 2024-09-13T09:26:55.988 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:55.989 INFO:teuthology.orchestra.run.smithi063.stdout:2838 writesame 0x12c0f76 thru 0x12cbe59 data_size 0x7b(0xaee4 bytes) 2024-09-13T09:26:55.990 INFO:teuthology.orchestra.run.smithi063.stdout:2839 write 0x536b157 thru 0x536f593 (0x443d bytes) 2024-09-13T09:26:55.992 INFO:teuthology.orchestra.run.smithi063.stdout:2840 writesame 0x3f89317 thru 0x3f8f17f data_size 0x1(0x5e69 bytes) 2024-09-13T09:26:56.018 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:56.011+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:56.018 INFO:teuthology.orchestra.run.smithi063.stdout:2842 punch from 0xa39e016 to 0xa3a89d7, (0xa9c1 bytes) 2024-09-13T09:26:56.019 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:56.019 INFO:teuthology.orchestra.run.smithi063.stdout:2844 writesame 0xa06f2c2 thru 0xa06f7e3 data_size 0x9(0x522 bytes) 2024-09-13T09:26:56.021 INFO:teuthology.orchestra.run.smithi063.stdout:2845 read 0xffa4f7 thru 0x10053da (0xaee4 bytes) 2024-09-13T09:26:56.026 INFO:teuthology.orchestra.run.smithi063.stdout:2846 punch from 0xc7676bf to 0xc76f4ae, (0x7def bytes) 2024-09-13T09:26:56.027 INFO:teuthology.orchestra.run.smithi063.stdout:2847 writesame 0x792641b thru 0x792fac4 data_size 0x46(0x96aa bytes) 2024-09-13T09:26:56.038 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:56.031+0000 7ff6e245c880 -1 Falling back to public interface 2024-09-13T09:26:56.039 INFO:teuthology.orchestra.run.smithi063.stdout:2848 writesame 0xd544982 thru 0xd54e28f data_size 0x89(0x990e bytes) 2024-09-13T09:26:56.042 INFO:teuthology.orchestra.run.smithi063.stdout:2849 writesame 0xc11d2a3 thru 0xc120e0b data_size 0x1(0x3b69 bytes) 2024-09-13T09:26:56.056 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:56.057 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:56.057 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:56.059 INFO:teuthology.orchestra.run.smithi063.stdout:2850 punch from 0xca571aa to 0xca58858, (0x16ae bytes) 2024-09-13T09:26:56.059 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:56.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:56.069 INFO:teuthology.orchestra.run.smithi063.stdout:2852 trunc from 0xdd55bbd to 0x9892836 2024-09-13T09:26:56.111 INFO:teuthology.orchestra.run.smithi063.stdout:2853 trunc from 0x9892836 to 0xc8a910c 2024-09-13T09:26:56.118 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:56.116+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:56.120 INFO:teuthology.orchestra.run.smithi063.stdout:2854 read 0xc46b219 thru 0xc46bc77 (0xa5f bytes) 2024-09-13T09:26:56.121 INFO:teuthology.orchestra.run.smithi063.stdout:2855 clone 24 order 19 su 4096 sc 10 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.809503759, 2024-09-13T09:26:56.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:26:56.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:26:56.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:26:56.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8094882999999999, 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.80947222, 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:26:56.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8094564470000005, 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:26:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8094345460000003, 2024-09-13T09:26:56.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:26:56.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:26:56.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8094186600000004, 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:26:56.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8094033449999998, 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:26:56.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8093873000000009, 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8093715760000002, 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8093569760000001, 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:26:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:26:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:26:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8093415480000008, 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:26:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.8093256619999991, 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:26:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:26:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:26:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85175 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.4602126870000003, 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048714652999999997, 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85175 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707870+0000", 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.131e-06 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707875+0000", 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9099999999999996e-06 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707879+0000", 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2690000000000001e-06 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.753863+0000", 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045983250000000003 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.754182+0000", 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031968300000000002 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756583+0000", 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024007339999999999 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.4601984189999992, 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:26:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:26:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.4600640519999999, 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:26:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.4600431609999998, 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:26:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97491 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 457728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1178458190000002, 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048679549000000003, 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97491 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050237+0000", 2024-09-13T09:26:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050243+0000", 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6740000000000002e-06 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050250+0000", 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4609999999999996e-06 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098540+0000", 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048290050000000001 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098608+0000", 2024-09-13T09:26:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8645000000000004e-05 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098915+0000", 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030641899999999999 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:26:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.085993556, 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0851159959999999, 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:26:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:26:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0850764359999996, 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.257 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:56.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:56.259 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:56.256+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:56.359 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:56.356+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:56.414 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:56.414 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:56.414 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:56.414 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:56.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:56.469 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:56.468+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:56.528 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:56.528 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:56.528 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:56.528 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:56.528 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:56.538 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:56.561 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:56.555+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.083984872999999, 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:26:56.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:26:56.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.273761610999999, 2024-09-13T09:26:56.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:26:56.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:26:56.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.657 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.01501874, 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:26:56.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.012816402, 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:56.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:26:56.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:26:56.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.012419801, 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:26:56.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:26:56.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:26:56.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.011544417, 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:26:56.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.674 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.011523434000001, 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.011472921999999, 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:26:56.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.011443063, 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:26:56.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:26:56.687 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9611033810000009, 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:26:56.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9610448030000001, 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:26:56.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:26:56.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9610221679999995, 2024-09-13T09:26:56.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:26:56.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:26:56.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9610052059999994, 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:26:56.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9601334759999993, 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:26:56.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9601008419999992, 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:26:56.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9600804430000007, 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:26:56.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9600387579999996, 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:26:56.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:26:56.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9599060969999993, 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9575556439999993, 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.9573716490000006, 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:26:56.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:56.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:26:56.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:56.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:56.736 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:56.737 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:26:56.738 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:56.736+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:56.795 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone23 intact 2024-09-13T09:26:56.838 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:56.836+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:56.856 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:26:56.857 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:56.866 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:26:56.939 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:56.931+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:57.000 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T09:26:57.021 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:57.022 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:57.022 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:57.022 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:57.022 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:57.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:26:57.040 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:57.036+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:57.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:26:57.140 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:57.136+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:57.143 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:57.143 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:26:57.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:26:57.143 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 74.178943294999996, 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:26:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:26:57.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:57.149 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.065723551, 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:26:57.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:57.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:57.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:26:57.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:26:57.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:26:57.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:26:57.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:26:57.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:26:57.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.065325536, 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:26:57.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:26:57.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.39021866, 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:57.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:26:57.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.390203727999999, 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:26:57.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:26:57.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.390170621999999, 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:26:57.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:26:57.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.390154329, 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:26:57.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.390138214, 2024-09-13T09:26:57.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:57.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:26:57.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:26:57.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:26:57.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.390107173000001, 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.390075550000001, 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:26:57.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.379182238, 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:26:57.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.379161901, 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:26:57.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:26:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:26:57.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:26:57.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:26:57.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:26:57.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:26:57.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.377127488999999, 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:57.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:26:57.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:26:57.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:26:57.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.377110433, 2024-09-13T09:26:57.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:26:57.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:26:57.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.377094265, 2024-09-13T09:26:57.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:57.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:26:57.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.376880603, 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:26:57.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:57.233 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.376861291999999, 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:26:57.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:26:57.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.238 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.376844899, 2024-09-13T09:26:57.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:26:57.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:26:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:26:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:26:57.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.245 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.376829237000001, 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:26:57.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:26:57.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:26:57.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:26:57.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:26:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:26:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:26:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.376802883, 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:26:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:26:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:26:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:26:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:26:57.258 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:57.259 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:57.260 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:57.260 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:57.251+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:57.261 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:26:57.267 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:57.291 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:57.323 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:57.353 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:57.359 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:57.351+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:57.384 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:26:57.385 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:57.385 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:57.385 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:57.385 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:57.386 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:57.386 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:57.386 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:57.386 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:57.386 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:57.389 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:57.389 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:57.428 INFO:teuthology.orchestra.run.smithi063.stdout:2856 writesame 0x17aa1e thru 0x185362 data_size 0x1(0xa945 bytes) 2024-09-13T09:26:57.473 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:57.467+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:57.504 INFO:teuthology.orchestra.run.smithi063.stdout:2857 trunc from 0xc8a910c to 0xb13c9b3 2024-09-13T09:26:57.561 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:57.556+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:57.571 INFO:teuthology.orchestra.run.smithi063.stdout:2858 trunc from 0xb13c9b3 to 0x6163459 2024-09-13T09:26:57.576 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:57.577 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:57.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:26:57.643 INFO:teuthology.orchestra.run.smithi063.stdout:2860 punch from 0x35daed to 0x35fa95, (0x1fa8 bytes) 2024-09-13T09:26:57.643 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:57.643 INFO:teuthology.orchestra.run.smithi063.stdout:2862 write 0x2a76582 thru 0x2a84aba (0xe539 bytes) 2024-09-13T09:26:57.662 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:57.655+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:57.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:26:57.705 INFO:teuthology.orchestra.run.smithi063.stdout:2863 punch from 0x522e3e to 0x52fe56, (0xd018 bytes) 2024-09-13T09:26:57.705 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:57.705 INFO:teuthology.orchestra.run.smithi063.stdout:2865 read 0x38d476c thru 0x38e2016 (0xd8ab bytes) 2024-09-13T09:26:57.727 INFO:teuthology.orchestra.run.smithi063.stdout:2867 write 0x22726c thru 0x22e8c1 (0x7656 bytes) 2024-09-13T09:26:57.744 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:57.744 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:57.762 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:57.760+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:57.764 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:26:57.775 INFO:teuthology.orchestra.run.smithi063.stdout:2868 write 0xe7a36d5 thru 0xe7a47f5 (0x1121 bytes) 2024-09-13T09:26:57.781 INFO:teuthology.orchestra.run.smithi063.stdout:2871 read 0x1fd8638 thru 0x1fe587f (0xd248 bytes) 2024-09-13T09:26:57.800 INFO:teuthology.orchestra.run.smithi063.stdout:2872 writesame 0x5ff004 thru 0x608232 data_size 0x1(0x922f bytes) 2024-09-13T09:26:57.863 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:57.860+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:57.886 INFO:teuthology.orchestra.run.smithi063.stdout:2873 write 0x5e9771 thru 0x5f1d79 (0x8609 bytes) 2024-09-13T09:26:57.891 INFO:teuthology.orchestra.run.smithi063.stdout:2874 write 0x4f823bd thru 0x4f8c426 (0xa06a bytes) 2024-09-13T09:26:57.931 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:57.931 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:57.944 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:26:57.948 INFO:teuthology.orchestra.run.smithi063.stdout:2876 read 0x1f080ce thru 0x1f17cd5 (0xfc08 bytes) 2024-09-13T09:26:57.950 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:57.951 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:57.951 INFO:teuthology.orchestra.run.smithi063.stdout:2883 writesame 0x992b9ff thru 0x9935589 data_size 0xd(0x9b8b bytes) 2024-09-13T09:26:57.964 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:57.955+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:57.971 INFO:teuthology.orchestra.run.smithi063.stdout:2884 write 0x5941fea thru 0x59463ca (0x43e1 bytes) 2024-09-13T09:26:58.009 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.009 INFO:teuthology.orchestra.run.smithi063.stdout:2886 writesame 0x7dc92fd thru 0x7dd280b data_size 0x1(0x950f bytes) 2024-09-13T09:26:58.027 INFO:teuthology.orchestra.run.smithi063.stdout:2887 writesame 0x6c8eab thru 0x6cb620 data_size 0x1(0x2776 bytes) 2024-09-13T09:26:58.031 INFO:teuthology.orchestra.run.smithi063.stdout:2889 write 0x80eefc2 thru 0x80fa5a9 (0xb5e8 bytes) 2024-09-13T09:26:58.035 INFO:teuthology.orchestra.run.smithi063.stdout:2890 write 0x893e01a thru 0x89410be (0x30a5 bytes) 2024-09-13T09:26:58.044 INFO:teuthology.orchestra.run.smithi063.stdout:2892 write 0x409bcf8 thru 0x40a090b (0x4c14 bytes) 2024-09-13T09:26:58.065 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:58.060+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:58.093 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:58.093 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:58.098 INFO:teuthology.orchestra.run.smithi063.stdout:2893 write 0x122e0f1 thru 0x1233cff (0x5c0f bytes) 2024-09-13T09:26:58.107 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:26:58.146 INFO:teuthology.orchestra.run.smithi063.stdout:2894 read 0x59a6fad thru 0x59b21ba (0xb20e bytes) 2024-09-13T09:26:58.157 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.157 INFO:teuthology.orchestra.run.smithi063.stdout:2896 writesame 0x7231e87 thru 0x7236555 data_size 0x1(0x46cf bytes) 2024-09-13T09:26:58.166 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:58.159+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:58.176 INFO:teuthology.orchestra.run.smithi063.stdout:2897 write 0xb275e46 thru 0xb27f23b (0x93f6 bytes) 2024-09-13T09:26:58.191 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.191 INFO:teuthology.orchestra.run.smithi063.stdout:2903 write 0xc7805f9 thru 0xc7856d7 (0x50df bytes) 2024-09-13T09:26:58.224 INFO:teuthology.orchestra.run.smithi063.stdout:2904 trunc from 0xe7a47f6 to 0x1688a03 2024-09-13T09:26:58.267 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:58.259+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:58.276 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:58.277 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:58.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:26:58.339 INFO:teuthology.orchestra.run.smithi063.stdout:2907 punch from 0x1035c6d to 0x103b7a0, (0x5b33 bytes) 2024-09-13T09:26:58.368 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:58.364+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:58.369 INFO:teuthology.orchestra.run.smithi063.stdout:2908 write 0xd3be002 thru 0xd3c4f9c (0x6f9b bytes) 2024-09-13T09:26:58.385 INFO:teuthology.orchestra.run.smithi063.stdout:2909 read 0xce58231 thru 0xce5ad22 (0x2af2 bytes) 2024-09-13T09:26:58.386 INFO:teuthology.orchestra.run.smithi063.stdout:2910 write 0xb50457b thru 0xb508d21 (0x47a7 bytes) 2024-09-13T09:26:58.397 INFO:teuthology.orchestra.run.smithi063.stdout:2911 write 0x1a85ba8 thru 0x1a8ed02 (0x915b bytes) 2024-09-13T09:26:58.419 INFO:teuthology.orchestra.run.smithi063.stdout:2912 punch from 0x317f93e to 0x318da9c, (0xe15e bytes) 2024-09-13T09:26:58.429 INFO:teuthology.orchestra.run.smithi063.stdout:2913 write 0xdd77b90 thru 0xdd879c7 (0xfe38 bytes) 2024-09-13T09:26:58.453 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.453 INFO:teuthology.orchestra.run.smithi063.stdout:2915 write 0x22672d0 thru 0x22718bf (0xa5f0 bytes) 2024-09-13T09:26:58.466 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:58.466 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:58.469 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:58.464+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:58.473 INFO:teuthology.orchestra.run.smithi063.stdout:2916 read 0x7f6e2b8 thru 0x7f73a63 (0x57ac bytes) 2024-09-13T09:26:58.473 INFO:teuthology.orchestra.run.smithi063.stdout:2917 punch from 0x63b5cd5 to 0x63bc408, (0x6733 bytes) 2024-09-13T09:26:58.473 INFO:teuthology.orchestra.run.smithi063.stdout:2918 read 0x90a72c5 thru 0x90b131e (0xa05a bytes) 2024-09-13T09:26:58.474 INFO:teuthology.orchestra.run.smithi063.stdout:2920 write 0x463bb80 thru 0x4642f3d (0x73be bytes) 2024-09-13T09:26:58.476 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:26:58.501 INFO:teuthology.orchestra.run.smithi063.stdout:2921 write 0xe077e43 thru 0xe07a0e5 (0x22a3 bytes) 2024-09-13T09:26:58.505 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.505 INFO:teuthology.orchestra.run.smithi063.stdout:2923 read 0x23077cd thru 0x230a46f (0x2ca3 bytes) 2024-09-13T09:26:58.505 INFO:teuthology.orchestra.run.smithi063.stdout:2924 write 0x5500583 thru 0x55087d4 (0x8252 bytes) 2024-09-13T09:26:58.526 INFO:teuthology.orchestra.run.smithi063.stdout:2925 punch from 0x847372a to 0x84773f1, (0x3cc7 bytes) 2024-09-13T09:26:58.526 INFO:teuthology.orchestra.run.smithi063.stdout:2926 read 0x17165bf thru 0x17173a7 (0xde9 bytes) 2024-09-13T09:26:58.527 INFO:teuthology.orchestra.run.smithi063.stdout:2927 read 0xa9c613c thru 0xa9cf64a (0x950f bytes) 2024-09-13T09:26:58.527 INFO:teuthology.orchestra.run.smithi063.stdout:2928 punch from 0x3bf046a to 0x3bf1625, (0x11bb bytes) 2024-09-13T09:26:58.527 INFO:teuthology.orchestra.run.smithi063.stdout:2932 writesame 0xb8262f3 thru 0xb82c936 data_size 0x77(0x6644 bytes) 2024-09-13T09:26:58.538 INFO:teuthology.orchestra.run.smithi063.stdout:2933 read 0x3f39cf8 thru 0x3f3daf3 (0x3dfc bytes) 2024-09-13T09:26:58.538 INFO:teuthology.orchestra.run.smithi063.stdout:2934 write 0x84ed869 thru 0x84fd250 (0xf9e8 bytes) 2024-09-13T09:26:58.558 INFO:teuthology.orchestra.run.smithi063.stdout:2935 write 0xb68b124 thru 0xb68e2fe (0x31db bytes) 2024-09-13T09:26:58.561 INFO:teuthology.orchestra.run.smithi063.stdout:2937 write 0x4daed96 thru 0x4db973e (0xa9a9 bytes) 2024-09-13T09:26:58.570 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:58.563+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:26:58.578 INFO:teuthology.orchestra.run.smithi063.stdout:2938 read 0x6c02840 thru 0x6c0afcb (0x878c bytes) 2024-09-13T09:26:58.579 INFO:teuthology.orchestra.run.smithi063.stdout:2939 read 0xa7224d8 thru 0xa723ef1 (0x1a1a bytes) 2024-09-13T09:26:58.579 INFO:teuthology.orchestra.run.smithi063.stdout:2940 writesame 0x2d27355 thru 0x2d2ce64 data_size 0x5e(0x5b10 bytes) 2024-09-13T09:26:58.589 INFO:teuthology.orchestra.run.smithi063.stdout:2941 read 0xb070a11 thru 0xb075082 (0x4672 bytes) 2024-09-13T09:26:58.593 INFO:teuthology.orchestra.run.smithi063.stdout:2942 read 0x3504a64 thru 0x350c662 (0x7bff bytes) 2024-09-13T09:26:58.593 INFO:teuthology.orchestra.run.smithi063.stdout:2946 read 0x684d247 thru 0x6853c0b (0x69c5 bytes) 2024-09-13T09:26:58.593 INFO:teuthology.orchestra.run.smithi063.stdout:2947 read 0x7bb5957 thru 0x7bb61b7 (0x861 bytes) 2024-09-13T09:26:58.593 INFO:teuthology.orchestra.run.smithi063.stdout:2948 punch from 0xb5741da to 0xb584079, (0xfe9f bytes) 2024-09-13T09:26:58.596 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:58.596 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:58.610 INFO:teuthology.orchestra.run.smithi063.stdout:2949 trunc from 0xe07a0e6 to 0x6ba7cf 2024-09-13T09:26:58.614 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:26:58.671 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:58.668+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:58.671 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:26:58.676 INFO:teuthology.orchestra.run.smithi063.stdout:2950 punch from 0x2b160c to 0x2bb4a1, (0x9e95 bytes) 2024-09-13T09:26:58.689 INFO:teuthology.orchestra.run.smithi063.stdout:2953 trunc from 0x6ba7cf to 0x8069196 2024-09-13T09:26:58.697 INFO:teuthology.orchestra.run.smithi063.stdout:2955 read 0x7fa8f2c thru 0x7fa9af8 (0xbcd bytes) 2024-09-13T09:26:58.697 INFO:teuthology.orchestra.run.smithi063.stdout:2956 writesame 0x50e0927 thru 0x50ea531 data_size 0x1(0x9c0b bytes) 2024-09-13T09:26:58.721 INFO:teuthology.orchestra.run.smithi063.stdout:2958 read 0x4e0fc59 thru 0x4e15339 (0x56e1 bytes) 2024-09-13T09:26:58.733 INFO:teuthology.orchestra.run.smithi063.stdout:2959 writesame 0x9fcaf97 thru 0x9fcf097 data_size 0x9(0x4101 bytes) 2024-09-13T09:26:58.734 INFO:teuthology.orchestra.run.smithi063.stdout:2961 write 0x798288b thru 0x7987869 (0x4fdf bytes) 2024-09-13T09:26:58.747 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.747 INFO:teuthology.orchestra.run.smithi063.stdout:2963 punch from 0x7d73906 to 0x7d7f618, (0xbd12 bytes) 2024-09-13T09:26:58.747 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:26:58.747 INFO:teuthology.orchestra.run.smithi063.stdout:2969 clone 25 order 25 su 2097152 sc 2 2024-09-13T09:26:58.772 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:58.768+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:58.774 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:58.774 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:58.784 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:26:58.873 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:58.868+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:58.900 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:58.900 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:58.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:26:58.974 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:26:58.967+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:26:59.028 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:59.028 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:59.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:26:59.075 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:59.072+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:59.145 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:59.145 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:59.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:26:59.201 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:26:59.196+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:26:59.276 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:26:59.272+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:26:59.308 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:59.308 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:59.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:26:59.377 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:59.376+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:59.391 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:26:59.468 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:59.469 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:59.478 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:26:59.479 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:59.476+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:59.541 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:26:59.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:26:59.579 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:26:59.571+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:26:59.586 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:59.586 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:59.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:26:59.680 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:59.676+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:59.705 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:26:59.705 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:26:59.715 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:26:59.745 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:26:59.739+0000 7ff6e245c880 -1 osd.3 46 log_to_monitors true 2024-09-13T09:26:59.781 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:26:59.776+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:26:59.786 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:26:59.794 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone24 from 0x9fcf098 (overlap 0x6ba7cf) to 0x2b0fec 2024-09-13T09:26:59.821 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:26:59.843 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:26:59.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:26:59.872 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:26:59.872 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:26:59.872 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:26:59.872 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:26:59.872 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:26:59.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:26:59.882 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:26:59.880+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:26:59.887 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:26:59.888 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:26:59.891 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:26:59.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:26:59.934 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T09:26:59.983 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:26:59.975+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:00.040 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.478413973000002, 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.668190710999999, 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.40944784, 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.407245502, 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.406848901, 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.405973517, 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.405952534000001, 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.405902021999999, 2024-09-13T09:27:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.405872163, 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.355532481000001, 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.355473903, 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.355451267999999, 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.355434305999999, 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.354562575999999, 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.354529941999999, 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:00.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:00.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.354509543000001, 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.354467858, 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.354335196999999, 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.119 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:00.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:00.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.351984743999999, 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:00.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:00.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.351800749000001, 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:00.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.132 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:00.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:00.134 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:00.127+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:00.234 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:00.232+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:00.244 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:00.244 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:00.244 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:00.244 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:00.254 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:00.316 INFO:teuthology.orchestra.run.smithi063.stdout:2971 read 0x709a08d thru 0x70a549d (0xb411 bytes) 2024-09-13T09:27:00.335 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:00.327+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:00.350 INFO:teuthology.orchestra.run.smithi063.stdout:2972 writesame 0xb20f061 thru 0xb21eccf data_size 0xd(0xfc6f bytes) 2024-09-13T09:27:00.357 INFO:teuthology.orchestra.run.smithi063.stdout:2973 write 0x6193275 thru 0x6194969 (0x16f5 bytes) 2024-09-13T09:27:00.364 INFO:teuthology.orchestra.run.smithi063.stdout:2974 trunc from 0xb21ecd0 to 0xcef5b77 2024-09-13T09:27:00.382 INFO:teuthology.orchestra.run.smithi063.stdout:2975 trunc from 0xcef5b77 to 0xad444ba 2024-09-13T09:27:00.403 INFO:teuthology.orchestra.run.smithi063.stdout:2976 read 0x342bd09 thru 0x342e89a (0x2b92 bytes) 2024-09-13T09:27:00.414 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:00.414 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:00.414 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:00.414 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:00.414 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:00.417 INFO:teuthology.orchestra.run.smithi063.stdout:2977 trunc from 0xad444ba to 0x6389dbe 2024-09-13T09:27:00.424 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:00.436 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:00.432+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:00.474 INFO:teuthology.orchestra.run.smithi063.stdout:2978 trunc from 0x6389dbe to 0x7fd0b65 2024-09-13T09:27:00.486 INFO:teuthology.orchestra.run.smithi063.stdout:2979 write 0x15ece40 thru 0x15f7e9c (0xb05d bytes) 2024-09-13T09:27:00.489 INFO:teuthology.orchestra.run.smithi063.stdout:2980 read 0x271e3ee thru 0x2722c5b (0x486e bytes) 2024-09-13T09:27:00.501 INFO:teuthology.orchestra.run.smithi063.stdout:2984 trunc from 0x7fd0b65 to 0x4893e6b 2024-09-13T09:27:00.534 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 77.570623509000001, 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.457403765, 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.45700575, 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781898873999999, 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781883942, 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781850836, 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781834543, 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781818427999999, 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781787387, 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.781755764, 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.770862451999999, 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.770842115000001, 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768807703, 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768790646999999, 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:00.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768774478999999, 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.598 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:00.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:00.602 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768560817000001, 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:00.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768541506, 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:00.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768525113000001, 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768509451, 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.616 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:00.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:00.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768483097000001, 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:00.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:00.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:00.624 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:00.626 INFO:teuthology.orchestra.run.smithi063.stdout:2985 writesame 0x7e1d47b thru 0x7e247fc data_size 0x5(0x7382 bytes) 2024-09-13T09:27:00.626 INFO:teuthology.orchestra.run.smithi063.stdout:2986 trunc from 0x7e247fd to 0x70f931 2024-09-13T09:27:00.626 INFO:teuthology.orchestra.run.smithi063.stdout:2987 trunc from 0x70f931 to 0x8187422 2024-09-13T09:27:00.627 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:00.624+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:00.627 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:00.644 INFO:teuthology.orchestra.run.smithi063.stdout:2990 writesame 0xde48ef1 thru 0xde55cf5 data_size 0x1(0xce05 bytes) 2024-09-13T09:27:00.645 INFO:teuthology.orchestra.run.smithi063.stdout:2992 writesame 0x6c8646b thru 0x6c8707d data_size 0x1(0xc13 bytes) 2024-09-13T09:27:00.651 INFO:teuthology.orchestra.run.smithi063.stdout:2994 punch from 0x8a776ef to 0x8a77a23, (0x334 bytes) 2024-09-13T09:27:00.651 INFO:teuthology.orchestra.run.smithi063.stdout:2995 punch from 0x7e6af29 to 0x7e7516d, (0xa244 bytes) 2024-09-13T09:27:00.651 INFO:teuthology.orchestra.run.smithi063.stdout:2996 read 0xa2f69d5 thru 0xa2f6e06 (0x432 bytes) 2024-09-13T09:27:00.651 INFO:teuthology.orchestra.run.smithi063.stdout:2997 write 0xa51e848 thru 0xa529cf3 (0xb4ac bytes) 2024-09-13T09:27:00.657 INFO:teuthology.orchestra.run.smithi063.stdout:2998 writesame 0x307240 thru 0x311540 data_size 0x1(0xa301 bytes) 2024-09-13T09:27:00.689 INFO:teuthology.orchestra.run.smithi063.stdout:2999 read 0x4f27e9 thru 0x500099 (0xd8b1 bytes) 2024-09-13T09:27:00.691 INFO:teuthology.orchestra.run.smithi063.stdout:3000 writesame 0x2f6f82f thru 0x2f70413 data_size 0x3(0xbe5 bytes) 2024-09-13T09:27:00.691 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:27:00.694 INFO:teuthology.orchestra.run.smithi063.stdout:3001 write 0xc1ae9c thru 0xc207b7 (0x591c bytes) 2024-09-13T09:27:00.695 INFO:teuthology.orchestra.run.smithi063.stdout:3002 punch from 0x408c0e2 to 0x408ea3f, (0x295d bytes) 2024-09-13T09:27:00.713 INFO:teuthology.orchestra.run.smithi063.stdout:3003 trunc from 0xde55cf6 to 0x3868b15 2024-09-13T09:27:00.727 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:00.719+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:00.733 INFO:teuthology.orchestra.run.smithi063.stdout:3004 read 0x35b83e6 thru 0x35c554d (0xd168 bytes) 2024-09-13T09:27:00.733 INFO:teuthology.orchestra.run.smithi063.stdout:3005 read 0x2aa6820 thru 0x2aa867e (0x1e5f bytes) 2024-09-13T09:27:00.734 INFO:teuthology.orchestra.run.smithi063.stdout:3006 read 0x362e901 thru 0x36316d2 (0x2dd2 bytes) 2024-09-13T09:27:00.735 INFO:teuthology.orchestra.run.smithi063.stdout:3007 read 0x351f93c thru 0x352dd20 (0xe3e5 bytes) 2024-09-13T09:27:00.737 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:00.731+0000 7ff6d5d4a700 -1 osd.3 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:27:00.751 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:00.751 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:00.751 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:00.751 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:00.755 INFO:teuthology.orchestra.run.smithi063.stdout:3010 write 0x820c06c thru 0x821bc9e (0xfc33 bytes) 2024-09-13T09:27:00.760 INFO:teuthology.orchestra.run.smithi063.stdout:3011 write 0x1a60945 thru 0x1a67cbe (0x737a bytes) 2024-09-13T09:27:00.761 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:00.763 INFO:teuthology.orchestra.run.smithi063.stdout:3012 read 0x3658f4f thru 0x365b190 (0x2242 bytes) 2024-09-13T09:27:00.764 INFO:teuthology.orchestra.run.smithi063.stdout:3014 read 0x3b10f7b thru 0x3b1df32 (0xcfb8 bytes) 2024-09-13T09:27:00.782 INFO:teuthology.orchestra.run.smithi063.stdout:3016 writesame 0xbaf6a1e thru 0xbb061ff data_size 0x1(0xf7e2 bytes) 2024-09-13T09:27:00.784 INFO:teuthology.orchestra.run.smithi063.stdout:3017 write 0x1e1c4b5 thru 0x1e28757 (0xc2a3 bytes) 2024-09-13T09:27:00.787 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:00.787 INFO:teuthology.orchestra.run.smithi063.stdout:3020 read 0x3281367 thru 0x328f94e (0xe5e8 bytes) 2024-09-13T09:27:00.789 INFO:teuthology.orchestra.run.smithi063.stdout:3021 clone 26 order 20 su 131072 sc 10 2024-09-13T09:27:00.820 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:00.827 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:00.824+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:00.886 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:00.887 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:00.887 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:00.887 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:00.887 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:00.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:00.928 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:00.919+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:00.942 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:00.942 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:00.942 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.009 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:01.009 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:01.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.441253802999999, 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.010 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:01.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:01.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:01.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:72716 3.12 3:4f8411ec:::rbd_data.11181839c212.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.199012755, 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0081225499999999992, 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.015 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 72716 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804331+0000", 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804537+0000", 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020520799999999999 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804563+0000", 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6067999999999998e-05 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804575+0000", 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2618e-05 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804679+0000", 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104003 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804949+0000", 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026972500000000001 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.805349+0000", 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039944600000000002 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.808978+0000", 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036290369999999999 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812305+0000", 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033272900000000001 2024-09-13T09:27:01.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812391+0000", 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6250999999999998e-05 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812456+0000", 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5324e-05 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87135 3.1b 3:d87e56f6:::rbd_data.11181839c212.0000000000000020:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.213368753999999, 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085077469999999995, 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87135 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789975+0000", 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:01.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789989+0000", 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790012+0000", 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2889999999999999e-05 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790034+0000", 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2221e-05 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790129+0000", 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5093000000000006e-05 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790376+0000", 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024744900000000001 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790600+0000", 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223654 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.791728+0000", 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011281170000000001 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798262+0000", 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065342899999999999 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798385+0000", 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012293199999999999 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798486+0000", 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100324 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87239 3.12 3:4ce3797a:::rbd_data.11181839c212.0000000000000133:head [set-alloc-hint object_size 262144 write_size 262144,write 144384~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.097989614999999, 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070822869999999996, 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87239 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905355+0000", 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905367+0000", 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1385e-05 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905394+0000", 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7627000000000002e-05 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905412+0000", 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7076999999999999e-05 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905512+0000", 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100139 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905698+0000", 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000186532 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905875+0000", 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000176751 2024-09-13T09:27:01.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.907019+0000", 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00114375 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912216+0000", 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051968719999999999 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912351+0000", 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013516099999999999 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912439+0000", 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8578000000000005e-05 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87861 3.1b 3:d9678466:::rbd_data.11181839c212.000000000000019b:head [set-alloc-hint object_size 262144 write_size 262144,write 179200~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:01.032 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.569241116000001, 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074187890000000003, 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87861 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:01.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434103+0000", 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434118+0000", 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4357e-05 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434154+0000", 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5816999999999999e-05 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434173+0000", 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9497e-05 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434266+0000", 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2987999999999998e-05 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434491+0000", 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000224441 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434675+0000", 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018440899999999999 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.436722+0000", 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020469669999999998 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441372+0000", 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046502059999999996 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441447+0000", 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4356000000000004e-05 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441524+0000", 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7818000000000006e-05 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87976 3.1b 3:dfd74b2f:::rbd_data.11181839c212.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.470641515000001, 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074366880000000003, 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87976 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:01.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532703+0000", 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532716+0000", 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532741+0000", 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5565999999999999e-05 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532756+0000", 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4367e-05 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532900+0000", 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014460699999999999 2024-09-13T09:27:01.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533180+0000", 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027956200000000002 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533377+0000", 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019749399999999999 2024-09-13T09:27:01.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535597+0000", 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022201439999999999 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.539992+0000", 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043942759999999999 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540079+0000", 2024-09-13T09:27:01.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6942999999999997e-05 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540142+0000", 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3192000000000001e-05 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87977 3.1 3:83505842:::rbd_data.11181839c212.000000000000027b:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:01.044 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.470539532, 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074050050000000001, 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87977 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532806+0000", 2024-09-13T09:27:01.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532815+0000", 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.346e-06 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532844+0000", 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8484e-05 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532853+0000", 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9700000000000005e-06 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532953+0000", 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100234 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533161+0000", 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020798600000000001 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533328+0000", 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016667299999999999 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535098+0000", 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017702589999999999 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540105+0000", 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050068170000000002 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540178+0000", 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3165999999999997e-05 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540212+0000", 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4175999999999999e-05 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:01.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.84202971, 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:01.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.733684521000001, 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:01.056 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:01.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:01.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.1953395669999995, 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:01.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.068 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.1951808120000003, 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:01.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.1951048719999999, 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:01.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.1941254289999996, 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:01.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:01.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.1865467089999999, 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.085 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:01.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:01.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:103778 3.1 3:868d9324:::rbd_data.11181839c212.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.3924848900000004, 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080962489999999998, 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 103778 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610860+0000", 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611180+0000", 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032002199999999999 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611217+0000", 2024-09-13T09:27:01.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6851999999999999e-05 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611235+0000", 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8003999999999999e-05 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611307+0000", 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1868999999999999e-05 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611544+0000", 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000237042 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611833+0000", 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028872000000000001 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.614498+0000", 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002665813 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618843+0000", 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043449370000000001 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618891+0000", 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7885e-05 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618958+0000", 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6874999999999999e-05 2024-09-13T09:27:01.096 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:103784 3.1f 3:fa4675b6:::rbd_data.11181839c212.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:52.624804+0000", 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.3785422500000006, 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0073385259999999997, 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.097 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 103784 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624804+0000", 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624804+0000", 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624803+0000", 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624940+0000", 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013690600000000001 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624959+0000", 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9045000000000001e-05 2024-09-13T09:27:01.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624971+0000", 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1542000000000001e-05 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.625042+0000", 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0885999999999999e-05 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.625220+0000", 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178264 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.625448+0000", 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022782400000000001 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.627655+0000", 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022075179999999999 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.632016+0000", 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043611040000000002 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.632091+0000", 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4525999999999998e-05 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.632143+0000", 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2055000000000002e-05 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.102 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.9198007449999999, 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:01.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:01.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.108 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:122815 3.b 3:d285db9c:::rbd_data.1118172bd520.0000000000000013:head [stat,delete] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.2071173850000001, 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007682538, 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 122815 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.109 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:01.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796233+0000", 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796242+0000", 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8009999999999993e-06 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796249+0000", 2024-09-13T09:27:01.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4800000000000004e-06 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801362+0000", 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051132110000000003 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801464+0000", 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101888 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801693+0000", 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022865299999999999 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803158+0000", 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001465343 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803777+0000", 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00061901299999999996 2024-09-13T09:27:01.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803829+0000", 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1662000000000001e-05 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803912+0000", 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2934000000000003e-05 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123058 3.1f 3:fceb619c:::rbd_data.1118172bd520.0000000000000007:1c [sparse-read 0~208896] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.59915544899999995, 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074653569999999997, 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123058 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404190+0000", 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404197+0000", 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0299999999999996e-06 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404214+0000", 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6612999999999998e-05 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404227+0000", 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.279e-05 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.410925+0000", 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066988780000000001 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.410985+0000", 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9876000000000001e-05 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411657+0000", 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00067126600000000001 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123059 3.1f 3:fceb619c:::rbd_data.1118172bd520.0000000000000007:1c [sparse-read 417792~106496] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.59910733500000002, 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079657760000000008, 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.119 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123059 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404238+0000", 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404246+0000", 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1899999999999998e-06 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404263+0000", 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7136999999999999e-05 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404275+0000", 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2697000000000001e-05 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411690+0000", 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007414464 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411751+0000", 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1230999999999997e-05 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.412205+0000", 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00045391099999999998 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.123 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.125 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:01.125 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:01.120+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:01.226 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:01.224+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:01.242 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:01.242 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:01.243 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:01.243 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.252 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:01.327 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:01.324+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:01.358 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:01.359 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:01.359 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:01.359 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:01.359 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.368 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:01.427 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:01.419+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:01.484 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.525989176000003, 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:01.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.833576535999999, 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:27:01.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.833455974, 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.495 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:01.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.360682790999999, 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:01.499 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:01.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930329795, 2024-09-13T09:27:01.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:01.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930303921, 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.514 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930278556999999, 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:01.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.518 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930193535000001, 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.519 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.93016014, 2024-09-13T09:27:01.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.527 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930146202, 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:01.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.532 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930108455999999, 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:01.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.93009054, 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.537 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930076892000001, 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.541 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:01.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:01.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:01.546 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.930010247, 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:01.550 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.929962186999999, 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.551 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.555 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.929904915, 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:01.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.929884806, 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.560 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:01.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.92986266, 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:01.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.929806393, 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:01.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:01.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:01.572 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.929790497000001, 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:01.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:01.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:01.576 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.578 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:01.578 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:01.571+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:01.679 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:01.671+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:01.712 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:27:01.745 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:01.745 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:01.745 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:01.745 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.755 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:01.779 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:01.771+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:01.860 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:01.860 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:01.860 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:01.871 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:01.892 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:01.917 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:01.911+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:01.981 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:01.975+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:02.032 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.223716834999998, 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:27:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.815540949999999, 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:27:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.815350827000003, 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:27:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.814538417999998, 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:27:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.625631769000002, 2024-09-13T09:27:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:27:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.060522384000002, 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.986983371000001, 2024-09-13T09:27:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.894440003, 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.068 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.866379426999998, 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:02.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.073 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.852121644, 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:02.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 30.383403190999999, 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:27:02.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.394478498000002, 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:02.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.189452450000001, 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.087 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:02.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.091114401999999, 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:27:02.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.090305796999999, 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.096 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.015713226999999, 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:02.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:02.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:02.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.821630784, 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:02.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.109 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.706009285, 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:27:02.113 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.437267943, 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:02.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:106878 3.4 e69/21 3:2395b47f:::rbd_data.1118fd5aef74.0000000000000000:head v 69'245, mlcod=68'243)", 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:02.118 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.0472928279999998, 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002931983, 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:02.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979362+0000", 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980127+0000", 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00076519900000000004 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980141+0000", 2024-09-13T09:27:02.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4419999999999999e-05 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980170+0000", 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8915e-05 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980224+0000", 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3940000000000002e-05 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:02.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980243+0000", 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9210000000000001e-05 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982244+0000", 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020012350000000001 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982295+0000", 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0101e-05 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:02.123 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:02.123 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:02.124 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:02.125 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:02.125 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:02.125 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:02.125 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:02.120+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:02.130 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:02.153 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:02.180 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:02.204 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:02.224 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:02.215+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:02.225 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:02.227 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:02.227 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:02.228 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:02.228 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-13T09:27:02.228 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:02.228 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:02.228 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:02.228 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:02.240 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.240 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:02.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:02.325 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:02.324+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:02.355 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.355 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:02.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:02.426 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:02.424+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:02.471 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.471 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:02.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:02.527 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:02.524+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:02.627 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:02.624+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:02.636 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.636 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:02.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:02.728 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:02.724+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:02.756 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.756 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:02.765 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:02.829 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:02.823+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:02.870 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.870 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:02.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:02.930 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:02.923+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:02.990 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:02.990 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.000 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:03.030 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:03.028+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:03.103 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.104 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:03.131 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:03.127+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:03.223 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.223 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:03.234 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:03.232+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:03.334 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:03.327+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:03.341 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.341 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:03.435 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:03.427+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:03.457 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.457 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:03.536 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:03.527+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:03.579 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.579 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:03.636 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:03.627+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:03.713 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.713 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.728 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:03.728 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:27:03.737 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:03.736+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:03.838 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone25 from 0xbb06200 (overlap 0x70f931) to 0x0 2024-09-13T09:27:03.838 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:03.836+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:03.853 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:03.853 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:03.866 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:03.939 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:03.931+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:03.980 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T09:27:04.002 INFO:teuthology.orchestra.run.smithi063.stdout:3022 trunc from 0xbb06200 to 0xb90195a 2024-09-13T09:27:04.006 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:04.007 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:04.018 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:04.061 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:04.055+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:04.065 INFO:teuthology.orchestra.run.smithi063.stdout:3023 write 0xcd4a8b1 thru 0xcd52e59 (0x85a9 bytes) 2024-09-13T09:27:04.070 INFO:teuthology.orchestra.run.smithi063.stdout:3025 write 0xb267525 thru 0xb2698aa (0x2386 bytes) 2024-09-13T09:27:04.079 INFO:teuthology.orchestra.run.smithi063.stdout:3026 read 0x6616c26 thru 0x661cb9c (0x5f77 bytes) 2024-09-13T09:27:04.094 INFO:teuthology.orchestra.run.smithi063.stdout:3027 write 0xda8b04e thru 0xda8ea54 (0x3a07 bytes) 2024-09-13T09:27:04.100 INFO:teuthology.orchestra.run.smithi063.stdout:3028 write 0x17bfe0c thru 0x17ca8c8 (0xaabd bytes) 2024-09-13T09:27:04.110 INFO:teuthology.orchestra.run.smithi063.stdout:3031 read 0xb7e7efa thru 0xb7f2762 (0xa869 bytes) 2024-09-13T09:27:04.112 INFO:teuthology.orchestra.run.smithi063.stdout:3032 punch from 0x890c7e6 to 0x8916e95, (0xa6af bytes) 2024-09-13T09:27:04.112 INFO:teuthology.orchestra.run.smithi063.stdout:3034 write 0x785b1e3 thru 0x7861739 (0x6557 bytes) 2024-09-13T09:27:04.118 INFO:teuthology.orchestra.run.smithi063.stdout:3037 write 0x270ebd5 thru 0x271997b (0xada7 bytes) 2024-09-13T09:27:04.124 INFO:teuthology.orchestra.run.smithi063.stdout:truncating to largest ever: 0xee25ec8 2024-09-13T09:27:04.126 INFO:teuthology.orchestra.run.smithi063.stdout:3038 trunc from 0xda8ea55 to 0xee25ec8 2024-09-13T09:27:04.135 INFO:teuthology.orchestra.run.smithi063.stdout:3039 write 0x49f9a8e thru 0x49fdd8f (0x4302 bytes) 2024-09-13T09:27:04.140 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:04.136+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:04.140 INFO:teuthology.orchestra.run.smithi063.stdout:3040 read 0x20dbb4f thru 0x20dd5e1 (0x1a93 bytes) 2024-09-13T09:27:04.141 INFO:teuthology.orchestra.run.smithi063.stdout:3041 read 0x31439ad thru 0x31462fa (0x294e bytes) 2024-09-13T09:27:04.142 INFO:teuthology.orchestra.run.smithi063.stdout:3042 write 0xab648b1 thru 0xab6c2c1 (0x7a11 bytes) 2024-09-13T09:27:04.154 INFO:teuthology.orchestra.run.smithi063.stdout:3043 write 0x43f7f00 thru 0x43fa5c8 (0x26c9 bytes) 2024-09-13T09:27:04.160 INFO:teuthology.orchestra.run.smithi063.stdout:3050 write 0x9e129f1 thru 0x9e16066 (0x3676 bytes) 2024-09-13T09:27:04.168 INFO:teuthology.orchestra.run.smithi063.stdout:3051 writesame 0x6f2275f thru 0x6f233ec data_size 0x1(0xc8e bytes) 2024-09-13T09:27:04.174 INFO:teuthology.orchestra.run.smithi063.stdout:3052 punch from 0xe2c583e to 0xe2c6c15, (0x13d7 bytes) 2024-09-13T09:27:04.174 INFO:teuthology.orchestra.run.smithi063.stdout:3055 writesame 0xce4e4f4 thru 0xce52788 data_size 0x7(0x4295 bytes) 2024-09-13T09:27:04.183 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:04.183 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:04.193 INFO:teuthology.orchestra.run.smithi063.stdout:3056 trunc from 0xee25ec8 to 0x908edf5 2024-09-13T09:27:04.197 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:04.239 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:04.240 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:04.236+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:04.246 INFO:teuthology.orchestra.run.smithi063.stdout:3057 trunc from 0x908edf5 to 0x828e63e 2024-09-13T09:27:04.287 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:04.287 INFO:teuthology.orchestra.run.smithi063.stdout:3059 flatten 2024-09-13T09:27:04.327 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:04.327 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:04.339 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:04.366 INFO:teuthology.orchestra.run.smithi063.stdout:3060 punch from 0x8ee287 to 0x8f504a, (0x6dc3 bytes) 2024-09-13T09:27:04.366 INFO:teuthology.orchestra.run.smithi063.stdout:3061 clone 27 order 19 su 262144 sc 11 2024-09-13T09:27:04.366 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:04.359+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:04.398 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:04.418 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:04.439 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:04.442 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:04.440+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:04.463 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:04.480 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.912246368000002, 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.481 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:04.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:72716 3.12 3:4f8411ec:::rbd_data.11181839c212.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.670005320000001, 2024-09-13T09:27:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0081225499999999992, 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 72716 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804331+0000", 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804537+0000", 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020520799999999999 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804563+0000", 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6067999999999998e-05 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804575+0000", 2024-09-13T09:27:04.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2618e-05 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804679+0000", 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104003 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804949+0000", 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026972500000000001 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.805349+0000", 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039944600000000002 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.808978+0000", 2024-09-13T09:27:04.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036290369999999999 2024-09-13T09:27:04.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812305+0000", 2024-09-13T09:27:04.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033272900000000001 2024-09-13T09:27:04.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812391+0000", 2024-09-13T09:27:04.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6250999999999998e-05 2024-09-13T09:27:04.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812456+0000", 2024-09-13T09:27:04.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5324e-05 2024-09-13T09:27:04.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.494 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.495 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87135 3.1b 3:d87e56f6:::rbd_data.11181839c212.0000000000000020:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:04.495 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:04.495 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.684361319000001, 2024-09-13T09:27:04.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085077469999999995, 2024-09-13T09:27:04.496 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.496 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87135 2024-09-13T09:27:04.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.498 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:04.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:04.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789975+0000", 2024-09-13T09:27:04.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:04.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789989+0000", 2024-09-13T09:27:04.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:27:04.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790012+0000", 2024-09-13T09:27:04.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2889999999999999e-05 2024-09-13T09:27:04.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790034+0000", 2024-09-13T09:27:04.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2221e-05 2024-09-13T09:27:04.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790129+0000", 2024-09-13T09:27:04.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5093000000000006e-05 2024-09-13T09:27:04.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790376+0000", 2024-09-13T09:27:04.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024744900000000001 2024-09-13T09:27:04.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790600+0000", 2024-09-13T09:27:04.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223654 2024-09-13T09:27:04.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.791728+0000", 2024-09-13T09:27:04.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011281170000000001 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798262+0000", 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065342899999999999 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798385+0000", 2024-09-13T09:27:04.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012293199999999999 2024-09-13T09:27:04.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798486+0000", 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100324 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87239 3.12 3:4ce3797a:::rbd_data.11181839c212.0000000000000133:head [set-alloc-hint object_size 262144 write_size 262144,write 144384~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.568982179999999, 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070822869999999996, 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87239 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905357+0000", 2024-09-13T09:27:04.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905355+0000", 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905367+0000", 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1385e-05 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905394+0000", 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7627000000000002e-05 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905412+0000", 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7076999999999999e-05 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905512+0000", 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100139 2024-09-13T09:27:04.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905698+0000", 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000186532 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.905875+0000", 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000176751 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.907019+0000", 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00114375 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912216+0000", 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051968719999999999 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912351+0000", 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013516099999999999 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.912439+0000", 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8578000000000005e-05 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.517 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87861 3.1b 3:d9678466:::rbd_data.11181839c212.000000000000019b:head [set-alloc-hint object_size 262144 write_size 262144,write 179200~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.040233681, 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074187890000000003, 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87861 2024-09-13T09:27:04.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434103+0000", 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434118+0000", 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4357e-05 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434154+0000", 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5816999999999999e-05 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434173+0000", 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9497e-05 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434266+0000", 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2987999999999998e-05 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434491+0000", 2024-09-13T09:27:04.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000224441 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434675+0000", 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018440899999999999 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.436722+0000", 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020469669999999998 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441372+0000", 2024-09-13T09:27:04.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046502059999999996 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441447+0000", 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4356000000000004e-05 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441524+0000", 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7818000000000006e-05 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87976 3.1b 3:dfd74b2f:::rbd_data.11181839c212.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.94163408, 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074366880000000003, 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87976 2024-09-13T09:27:04.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532703+0000", 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532716+0000", 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532741+0000", 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5565999999999999e-05 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532756+0000", 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4367e-05 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532900+0000", 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014460699999999999 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533180+0000", 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027956200000000002 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533377+0000", 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019749399999999999 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535597+0000", 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022201439999999999 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.539992+0000", 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043942759999999999 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540079+0000", 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6942999999999997e-05 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540142+0000", 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3192000000000001e-05 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87977 3.1 3:83505842:::rbd_data.11181839c212.000000000000027b:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.941532097, 2024-09-13T09:27:04.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074050050000000001, 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87977 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532806+0000", 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532815+0000", 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.346e-06 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532844+0000", 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8484e-05 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532853+0000", 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9700000000000005e-06 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532953+0000", 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100234 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533161+0000", 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020798600000000001 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533328+0000", 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016667299999999999 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535098+0000", 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017702589999999999 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540105+0000", 2024-09-13T09:27:04.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050068170000000002 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540178+0000", 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3165999999999997e-05 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540212+0000", 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4175999999999999e-05 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.313022275, 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:04.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:04.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.204677086, 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:04.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:04.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:04.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.666332132000001, 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:04.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:04.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:04.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.666173377, 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.555 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:04.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:04.561 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.666097436999999, 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.562 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.568 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.665117993999999, 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:04.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:04.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:04.574 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.574 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.657539273999999, 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.575 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:04.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:04.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:04.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:04.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:04.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:04.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:04.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:04.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:04.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:04.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:04.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:04.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:04.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:04.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:04.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:04.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:04.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:04.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.587 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.587 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:103778 3.1 3:868d9324:::rbd_data.11181839c212.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:04.588 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:04.588 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 11.863477455, 2024-09-13T09:27:04.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080962489999999998, 2024-09-13T09:27:04.588 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.588 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.589 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 103778 2024-09-13T09:27:04.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.589 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:04.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610860+0000", 2024-09-13T09:27:04.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:04.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611180+0000", 2024-09-13T09:27:04.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032002199999999999 2024-09-13T09:27:04.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611217+0000", 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6851999999999999e-05 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611235+0000", 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8003999999999999e-05 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611307+0000", 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1868999999999999e-05 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611544+0000", 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000237042 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611833+0000", 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028872000000000001 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.614498+0000", 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002665813 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618843+0000", 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043449370000000001 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618891+0000", 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7885e-05 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618958+0000", 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6874999999999999e-05 2024-09-13T09:27:04.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:103784 3.1f 3:fa4675b6:::rbd_data.11181839c212.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:52.624804+0000", 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 11.849534815, 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0073385259999999997, 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 103784 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624804+0000", 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624804+0000", 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624803+0000", 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624940+0000", 2024-09-13T09:27:04.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013690600000000001 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624959+0000", 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9045000000000001e-05 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.624971+0000", 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1542000000000001e-05 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.625042+0000", 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0885999999999999e-05 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.625220+0000", 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178264 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.625448+0000", 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022782400000000001 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.627655+0000", 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022075179999999999 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.632016+0000", 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043611040000000002 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.632091+0000", 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4525999999999998e-05 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.632143+0000", 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2055000000000002e-05 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.3907933099999994, 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.604 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:04.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:04.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:04.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:04.610 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:122815 3.b 3:d285db9c:::rbd_data.1118172bd520.0000000000000013:head [stat,delete] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6781099499999996, 2024-09-13T09:27:04.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007682538, 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 122815 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:04.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796233+0000", 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796242+0000", 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8009999999999993e-06 2024-09-13T09:27:04.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796249+0000", 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4800000000000004e-06 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801362+0000", 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051132110000000003 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801464+0000", 2024-09-13T09:27:04.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101888 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801693+0000", 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022865299999999999 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803158+0000", 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001465343 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803777+0000", 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00061901299999999996 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803829+0000", 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1662000000000001e-05 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803912+0000", 2024-09-13T09:27:04.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2934000000000003e-05 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123058 3.1f 3:fceb619c:::rbd_data.1118172bd520.0000000000000007:1c [sparse-read 0~208896] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.0701480139999999, 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074653569999999997, 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.618 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123058 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404190+0000", 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404197+0000", 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0299999999999996e-06 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404214+0000", 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6612999999999998e-05 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404227+0000", 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.279e-05 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.410925+0000", 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066988780000000001 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.410985+0000", 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9876000000000001e-05 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411657+0000", 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00067126600000000001 2024-09-13T09:27:04.622 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123059 3.1f 3:fceb619c:::rbd_data.1118172bd520.0000000000000007:1c [sparse-read 417792~106496] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.0700998999999998, 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079657760000000008, 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:04.623 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123059 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404238+0000", 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404246+0000", 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1899999999999998e-06 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404263+0000", 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7136999999999999e-05 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404275+0000", 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2697000000000001e-05 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411690+0000", 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007414464 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411751+0000", 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1230999999999997e-05 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.412205+0000", 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00045391099999999998 2024-09-13T09:27:04.627 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.628 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.628 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.628 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:04.628 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:04.628 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:04.629 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:04.630 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:04.630 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:04.631 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:04.631 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:04.631 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:04.631 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:04.633 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:04.628+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:04.730 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:04.728+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:04.741 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:27:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:04.799 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:04.799 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:04.799 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:04.810 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:04.831 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:04.828+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:04.917 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:04.917 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:04.917 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:04.917 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:04.917 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:04.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:04.931 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:04.923+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:05.033 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:05.028+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:05.041 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:05.041 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.083153373000002, 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:05.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:05.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:05.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:05.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.390740733000001, 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:27:05.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:05.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:27:05.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.390620170999998, 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:05.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:05.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.917846988000001, 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:05.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.062 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487493992000001, 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:05.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:05.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487468117999999, 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.068 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487442754, 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:05.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487357732, 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:05.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487324337, 2024-09-13T09:27:05.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:05.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487310398999998, 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:05.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:05.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487272653000002, 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:05.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:05.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487254737000001, 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:05.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:05.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487241089000001, 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:05.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487174444000001, 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:05.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:05.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:05.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487126384, 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.112 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487069112, 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:05.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.121 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487049002999999, 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:05.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.487026857, 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:05.126 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:05.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.130 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.486970589999999, 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.131 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:05.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:05.136 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.486954694000001, 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:05.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.141 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.141 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:05.142 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:05.143 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:05.135+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:05.243 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:05.240+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:05.304 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:05.304 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:05.304 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:05.305 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:05.314 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:05.344 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:05.344+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:05.425 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:05.425 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:05.425 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:05.425 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:05.425 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:05.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:05.445 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:05.444+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:05.544 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:05.544 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:05.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:05.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:05.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.736733799999996, 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:27:05.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:27:05.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.328557914999998, 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:05.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:27:05.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.553 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.328367792000002, 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:05.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:27:05.557 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.327555383000004, 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.138648734, 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:27:05.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:27:05.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.573539349000001, 2024-09-13T09:27:05.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.500000335999999, 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:05.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.407456967999998, 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:27:05.575 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:27:05.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:27:05.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.379396391999997, 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:05.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.583 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.365138608999999, 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:05.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.896420155999998, 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.588 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:05.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:27:05.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.592 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.907495463, 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:05.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.702469414999999, 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:05.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.601 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.604131367000001, 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:05.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:27:05.605 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.603322762000001, 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:27:05.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:27:05.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.528730192000001, 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:05.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.334647748999998, 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:05.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.618 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.219026249999999, 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:27:05.622 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.950284908, 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:05.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:05.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:106878 3.4 e69/21 3:2395b47f:::rbd_data.1118fd5aef74.0000000000000000:head v 69'245, mlcod=68'243)", 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.5603097930000001, 2024-09-13T09:27:05.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002931983, 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:05.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979362+0000", 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980127+0000", 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00076519900000000004 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:05.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980141+0000", 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4419999999999999e-05 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980170+0000", 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8915e-05 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980224+0000", 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3940000000000002e-05 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980243+0000", 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9210000000000001e-05 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982244+0000", 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020012350000000001 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982295+0000", 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0101e-05 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.631 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.632 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:05.632 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:05.632 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:05.633 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:05.634 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:05.627+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:05.734 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:05.727+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:05.762 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:05.762 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:05.762 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:05.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:05.835 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:05.827+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:05.890 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:05.890 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:05.891 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:05.891 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:05.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:05.936 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:05.927+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:05.959 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:05.959 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:05.959 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:27:05.959 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:27:05.972 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T09:27:05.990 INFO:teuthology.orchestra.run.smithi063.stdout:3063 write 0x41840f8 thru 0x4185766 (0x166f bytes) 2024-09-13T09:27:06.004 INFO:teuthology.orchestra.run.smithi063.stdout:3065 read 0x583f512 thru 0x584e59e (0xf08d bytes) 2024-09-13T09:27:06.004 INFO:teuthology.orchestra.run.smithi063.stdout:3067 read 0x1aaf7ce thru 0x1ab5b18 (0x634b bytes) 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307698707, 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307676192999999, 2024-09-13T09:27:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307660356, 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307463412000001, 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307439087999999, 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307422628000001, 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307406607000001, 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307356274, 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.307252698999999, 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297874903, 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:06.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:06.050 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297856239000001, 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:06.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297839749000001, 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:06.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:06.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297823963999999, 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:06.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:06.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:06.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:06.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297807843000001, 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:06.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297791938, 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:06.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297775469000001, 2024-09-13T09:27:06.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:06.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297753617000001, 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297736801999999, 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:06.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297720505000001, 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:06.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.297704904, 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.095 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:06.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:3068 write 0xa6e14e3 thru 0xa6eff8a (0xeaa8 bytes) 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:3071 read 0x2d05966 thru 0x2d0a4c3 (0x4b5e bytes) 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:3073 punch from 0x6f6b294 to 0x6f78b5b, (0xd8c7 bytes) 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:3074 read 0x584cb2d thru 0x5852226 (0x56fa bytes) 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:3076 trunc from 0xa6eff8b to 0x5403af 2024-09-13T09:27:06.098 INFO:teuthology.orchestra.run.smithi063.stdout:3077 write 0x2beae3b thru 0x2bf5c87 (0xae4d bytes) 2024-09-13T09:27:06.099 INFO:teuthology.orchestra.run.smithi063.stdout:3078 writesame 0xa7c350 thru 0xa7f090 data_size 0x7(0x2d41 bytes) 2024-09-13T09:27:06.099 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:06.087+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:06.100 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.101 INFO:teuthology.orchestra.run.smithi063.stdout:3080 punch from 0x441d8f to 0x442461, (0x6d2 bytes) 2024-09-13T09:27:06.108 INFO:teuthology.orchestra.run.smithi063.stdout:3082 trunc from 0x2bf5c88 to 0x6ada876 2024-09-13T09:27:06.117 INFO:teuthology.orchestra.run.smithi063.stdout:3084 punch from 0x10c5e48 to 0x10d47e6, (0xe99e bytes) 2024-09-13T09:27:06.117 INFO:teuthology.orchestra.run.smithi063.stdout:3086 punch from 0x8ffbe5 to 0x90e446, (0xe861 bytes) 2024-09-13T09:27:06.117 INFO:teuthology.orchestra.run.smithi063.stdout:3089 read 0x3498fb9 thru 0x349e75b (0x57a3 bytes) 2024-09-13T09:27:06.117 INFO:teuthology.orchestra.run.smithi063.stdout:3090 writesame 0x10b5983 thru 0x10bf1d1 data_size 0x3(0x984f bytes) 2024-09-13T09:27:06.131 INFO:teuthology.orchestra.run.smithi063.stdout:3091 write 0xa4b771 thru 0xa4caf2 (0x1382 bytes) 2024-09-13T09:27:06.134 INFO:teuthology.orchestra.run.smithi063.stdout:3092 flatten 2024-09-13T09:27:06.165 INFO:teuthology.orchestra.run.smithi063.stdout:3093 write 0x4a3c46c thru 0x4a45d29 (0x98be bytes) 2024-09-13T09:27:06.170 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.170 INFO:teuthology.orchestra.run.smithi063.stdout:3096 punch from 0x3b7e15d to 0x3b88aa8, (0xa94b bytes) 2024-09-13T09:27:06.171 INFO:teuthology.orchestra.run.smithi063.stdout:3097 read 0x3639502 thru 0x363a6e4 (0x11e3 bytes) 2024-09-13T09:27:06.186 INFO:teuthology.orchestra.run.smithi063.stdout:3099 writesame 0xa5ed7a3 thru 0xa5f9cb2 data_size 0x18(0xc510 bytes) 2024-09-13T09:27:06.187 INFO:teuthology.orchestra.run.smithi063.stdout:3100 write 0x80c20a9 thru 0x80c7567 (0x54bf bytes) 2024-09-13T09:27:06.192 INFO:teuthology.orchestra.run.smithi063.stdout:3101 write 0x96ca177 thru 0x96d9b5b (0xf9e5 bytes) 2024-09-13T09:27:06.197 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:06.191+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:06.198 INFO:teuthology.orchestra.run.smithi063.stdout:3102 punch from 0x71747b5 to 0x7175f2d, (0x1778 bytes) 2024-09-13T09:27:06.198 INFO:teuthology.orchestra.run.smithi063.stdout:3104 read 0x2cdddd9 thru 0x2ce28f1 (0x4b19 bytes) 2024-09-13T09:27:06.198 INFO:teuthology.orchestra.run.smithi063.stdout:3105 read 0x70e6482 thru 0x70e99d7 (0x3556 bytes) 2024-09-13T09:27:06.199 INFO:teuthology.orchestra.run.smithi063.stdout:3106 read 0x3e4be0f thru 0x3e50a72 (0x4c64 bytes) 2024-09-13T09:27:06.199 INFO:teuthology.orchestra.run.smithi063.stdout:3108 read 0x2b69f9b thru 0x2b709ac (0x6a12 bytes) 2024-09-13T09:27:06.199 INFO:teuthology.orchestra.run.smithi063.stdout:3109 read 0x95fd658 thru 0x960cd85 (0xf72e bytes) 2024-09-13T09:27:06.212 INFO:teuthology.orchestra.run.smithi063.stdout:3112 writesame 0xbce1da9 thru 0xbceccc9 data_size 0x1(0xaf21 bytes) 2024-09-13T09:27:06.212 INFO:teuthology.orchestra.run.smithi063.stdout:3113 punch from 0x3ab7668 to 0x3ac40f6, (0xca8e bytes) 2024-09-13T09:27:06.212 INFO:teuthology.orchestra.run.smithi063.stdout:3115 read 0x447c03 thru 0x452908 (0xad06 bytes) 2024-09-13T09:27:06.213 INFO:teuthology.orchestra.run.smithi063.stdout:3116 write 0xf2f6dc thru 0xf3c6d4 (0xcff9 bytes) 2024-09-13T09:27:06.219 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:06.219 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:06.219 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:06.220 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.220 INFO:teuthology.orchestra.run.smithi063.stdout:3118 read 0x6292acb thru 0x6294477 (0x19ad bytes) 2024-09-13T09:27:06.224 INFO:teuthology.orchestra.run.smithi063.stdout:3119 trunc from 0xbceccca to 0xdd476c3 2024-09-13T09:27:06.229 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:06.233 INFO:teuthology.orchestra.run.smithi063.stdout:3120 write 0x5cb2623 thru 0x5cc14da (0xeeb8 bytes) 2024-09-13T09:27:06.242 INFO:teuthology.orchestra.run.smithi063.stdout:3121 read 0x25dcc3e thru 0x25ea37e (0xd741 bytes) 2024-09-13T09:27:06.242 INFO:teuthology.orchestra.run.smithi063.stdout:3122 write 0x3e07c6b thru 0x3e0cc0b (0x4fa1 bytes) 2024-09-13T09:27:06.247 INFO:teuthology.orchestra.run.smithi063.stdout:3123 write 0x8c1582d thru 0x8c202da (0xaaae bytes) 2024-09-13T09:27:06.253 INFO:teuthology.orchestra.run.smithi063.stdout:3124 read 0x3297606 thru 0x32a190e (0xa309 bytes) 2024-09-13T09:27:06.253 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.253 INFO:teuthology.orchestra.run.smithi063.stdout:3126 write 0x4eca513 thru 0x4ecabbb (0x6a9 bytes) 2024-09-13T09:27:06.267 INFO:teuthology.orchestra.run.smithi063.stdout:3127 write 0xde3088d thru 0xde405e0 (0xfd54 bytes) 2024-09-13T09:27:06.276 INFO:teuthology.orchestra.run.smithi063.stdout:3129 read 0xd583718 thru 0xd58de25 (0xa70e bytes) 2024-09-13T09:27:06.276 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.276 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.276 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.276 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.277 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:06.277 INFO:teuthology.orchestra.run.smithi063.stdout:3135 punch from 0x27c18fb to 0x27ce849, (0xcf4e bytes) 2024-09-13T09:27:06.288 INFO:teuthology.orchestra.run.smithi063.stdout:3136 writesame 0xe41f073 thru 0xe423876 data_size 0x16(0x4804 bytes) 2024-09-13T09:27:06.288 INFO:teuthology.orchestra.run.smithi063.stdout:3138 write 0x7e43eac thru 0x7e4ab10 (0x6c65 bytes) 2024-09-13T09:27:06.294 INFO:teuthology.orchestra.run.smithi063.stdout:3139 write 0x603aed8 thru 0x603ff0f (0x5038 bytes) 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 78, 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout: "fsid": "eb7f03c3-9691-4766-8999-6f4f526cda71", 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout: "created": "2024-09-13T09:25:30.379173+0000", 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout: "modified": "2024-09-13T09:27:05.719604+0000", 2024-09-13T09:27:06.298 INFO:teuthology.orchestra.run.smithi063.stdout: "last_up_change": "2024-09-13T09:27:01.686116+0000", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "last_in_change": "2024-09-13T09:26:38.515771+0000", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_num": 5801984, 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_set": [ 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "noscrub", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "pglog_hardlimit", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "purged_snapdirs", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_deletes", 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "sortbitwise" 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_version": 6, 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:27:06.299 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_snapshot": "", 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_max": 3, 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd": 8, 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "min_compat_client": "jewel", 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "require_osd_release": "reef", 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_crimson": false, 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pools": [ 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 1, 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "rbd", 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:41.357433+0000", 2024-09-13T09:27:06.300 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 8, 2024-09-13T09:27:06.301 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 8, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 8, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 8, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 8, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:06.302 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 3, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 19, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:06.303 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:06.304 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:06.305 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 2, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 2, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 1, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 2, 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": ".mgr", 2024-09-13T09:27:06.306 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:42.578616+0000", 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool", 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:06.307 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 1, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 1, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 1, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 1, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 1, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:06.308 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 0, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 0, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:06.309 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:06.310 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "options": { 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_max": 32, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_min": 1 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.311 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": {} 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 8, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 8, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 0.25, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 3, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "pool_client.0", 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:51.228779+0000", 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:06.313 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 32, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 32, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 32, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 32, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 32, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:06.314 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "77", 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 30, 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 77, 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:06.315 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:06.316 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:06.317 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 1.5, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 1.5, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 1, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 1.5, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 1.5, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:06.318 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "osds": [ 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "54ea2bd8-55b0-4ded-8a16-041261c5ba5a", 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.319 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 76, 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6816", 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6817", 2024-09-13T09:27:06.320 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6818", 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.321 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6819", 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6822", 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.322 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6823", 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6820", 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6821", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6817/2906107514", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6819/2906107514", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6823/2906107514", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6821/2906107514", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.324 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "0c95aee9-f871-4f6f-bbbd-1206db5d331c", 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.325 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6824", 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6825", 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6826", 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6827", 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6830", 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6831", 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.328 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6828", 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6829", 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6825/3622854518", 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6827/3622854518", 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6831/3622854518", 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6829/3622854518", 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "f8253db2-d4da-46b5-95cf-cd5c9ebd7328", 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.330 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 14, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 0, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6800", 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6801", 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6802", 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6803", 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6806", 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6807", 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6804", 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6805", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6801/2105659441", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6803/2105659441", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6807/2105659441", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6805/2105659441", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "67c11a52-baeb-4ea8-89ea-26018ffdc35a", 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 15, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 46, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 74, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 48, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6808", 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6809", 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6810", 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6811", 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6814", 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6815", 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.339 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6812", 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6813", 2024-09-13T09:27:06.340 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6809/4293004354", 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6811/4293004354", 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6815/4293004354", 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6813/4293004354", 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.341 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "1cec9702-fa09-4d7c-b111-7d10b1950cfe", 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 60, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6800", 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6801", 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6802", 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6803", 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.344 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6806", 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6807", 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6804", 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6805", 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6801/1789187713", 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6803/1789187713", 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1789187713", 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1789187713", 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "7c0b908a-65c9-426c-867c-cc8ed65084b3", 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.347 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6809", 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6811", 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6813", 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.349 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6815", 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6821", 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6823", 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6817", 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.351 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6819", 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6811/3410135021", 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6815/3410135021", 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3410135021", 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6819/3410135021", 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "78d5990a-4535-4110-ba04-88f01ab9fe28", 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 76, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.353 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6808", 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6810", 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6812", 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6814", 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6820", 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6822", 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.356 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6816", 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6818", 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.357 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6810/4152410871", 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6814/4152410871", 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6822/4152410871", 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6818/4152410871", 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "53f8511c-729f-4423-8eac-813c01f8f72b", 2024-09-13T09:27:06.358 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6824", 2024-09-13T09:27:06.359 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6825", 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6826", 2024-09-13T09:27:06.360 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6827", 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.363 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6830", 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6831", 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6828", 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6829", 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.365 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6825/2923799929", 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6827/2923799929", 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2923799929", 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2923799929", 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_xinfo": [ 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.366 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.267895+0000", 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.367 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.161313+0000", 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.368 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.447964+0000", 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.021320+0000", 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 48 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.369 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.883961+0000", 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.370 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.938146+0000", 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.627237+0000", 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.887523+0000", 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap": [], 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_items": [], 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_primaries": [], 2024-09-13T09:27:06.372 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_temp": [], 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_temp": [], 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklist": {}, 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "range_blocklist": {}, 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profiles": { 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "default": { 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-failure-domain": "osd", 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "k": "2", 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "m": "1", 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "plugin": "jerasure", 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "technique": "reed_sol_van" 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps_queue": [], 2024-09-13T09:27:06.373 INFO:teuthology.orchestra.run.smithi063.stdout: "new_removed_snaps": [], 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "new_purged_snaps": [], 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_node_flags": {}, 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "device_class_flags": {}, 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode": { 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_enabled": false, 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_bucket_count": 0, 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_bucket": 0 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:06.374 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:06.375 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:06.296+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:06.375 INFO:teuthology.orchestra.run.smithi063.stdout:3140 writesame 0x3e53c9c thru 0x3e5f567 data_size 0x2(0xb8cc bytes) 2024-09-13T09:27:06.376 INFO:teuthology.orchestra.run.smithi063.stdout:3142 writesame 0x714e438 thru 0x7154d08 data_size 0x1(0x68d1 bytes) 2024-09-13T09:27:06.377 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.377 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:06.377 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:06.377 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:06.377 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:06.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:06.379 INFO:tasks.thrashosds.thrasher:j is {'epoch': 78, 'fsid': 'eb7f03c3-9691-4766-8999-6f4f526cda71', 'created': '2024-09-13T09:25:30.379173+0000', 'modified': '2024-09-13T09:27:05.719604+0000', 'last_up_change': '2024-09-13T09:27:01.686116+0000', 'last_in_change': '2024-09-13T09:26:38.515771+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:25:41.357433+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:25:42.578616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:25:51.228779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 30, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '54ea2bd8-55b0-4ded-8a16-041261c5ba5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6816', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6817', 'nonce': 2906107514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6818', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6819', 'nonce': 2906107514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6822', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6823', 'nonce': 2906107514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6820', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6821', 'nonce': 2906107514}]}, 'public_addr': '172.21.15.63:6817/2906107514', 'cluster_addr': '172.21.15.63:6819/2906107514', 'heartbeat_back_addr': '172.21.15.63:6823/2906107514', 'heartbeat_front_addr': '172.21.15.63:6821/2906107514', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0c95aee9-f871-4f6f-bbbd-1206db5d331c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6824', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6825', 'nonce': 3622854518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6826', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6827', 'nonce': 3622854518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6830', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6831', 'nonce': 3622854518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6828', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6829', 'nonce': 3622854518}]}, 'public_addr': '172.21.15.63:6825/3622854518', 'cluster_addr': '172.21.15.63:6827/3622854518', 'heartbeat_back_addr': '172.21.15.63:6831/3622854518', 'heartbeat_front_addr': '172.21.15.63:6829/3622854518', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f8253db2-d4da-46b5-95cf-cd5c9ebd7328', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6800', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6801', 'nonce': 2105659441}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6802', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6803', 'nonce': 2105659441}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6806', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6807', 'nonce': 2105659441}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6804', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6805', 'nonce': 2105659441}]}, 'public_addr': '172.21.15.63:6801/2105659441', 'cluster_addr': '172.21.15.63:6803/2105659441', 'heartbeat_back_addr': '172.21.15.63:6807/2105659441', 'heartbeat_front_addr': '172.21.15.63:6805/2105659441', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '67c11a52-baeb-4ea8-89ea-26018ffdc35a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 46, 'up_from': 74, 'up_thru': 77, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6808', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6809', 'nonce': 4293004354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6810', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6811', 'nonce': 4293004354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6814', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6815', 'nonce': 4293004354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6812', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6813', 'nonce': 4293004354}]}, 'public_addr': '172.21.15.63:6809/4293004354', 'cluster_addr': '172.21.15.63:6811/4293004354', 'heartbeat_back_addr': '172.21.15.63:6815/4293004354', 'heartbeat_front_addr': '172.21.15.63:6813/4293004354', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '1cec9702-fa09-4d7c-b111-7d10b1950cfe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1789187713}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1789187713}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1789187713}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1789187713}]}, 'public_addr': '172.21.15.99:6801/1789187713', 'cluster_addr': '172.21.15.99:6803/1789187713', 'heartbeat_back_addr': '172.21.15.99:6807/1789187713', 'heartbeat_front_addr': '172.21.15.99:6805/1789187713', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7c0b908a-65c9-426c-867c-cc8ed65084b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6809', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3410135021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6813', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3410135021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6821', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3410135021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6817', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3410135021}]}, 'public_addr': '172.21.15.99:6811/3410135021', 'cluster_addr': '172.21.15.99:6815/3410135021', 'heartbeat_back_addr': '172.21.15.99:6823/3410135021', 'heartbeat_front_addr': '172.21.15.99:6819/3410135021', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78d5990a-4535-4110-ba04-88f01ab9fe28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6810', 'nonce': 4152410871}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6814', 'nonce': 4152410871}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6822', 'nonce': 4152410871}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6818', 'nonce': 4152410871}]}, 'public_addr': '172.21.15.99:6810/4152410871', 'cluster_addr': '172.21.15.99:6814/4152410871', 'heartbeat_back_addr': '172.21.15.99:6822/4152410871', 'heartbeat_front_addr': '172.21.15.99:6818/4152410871', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '53f8511c-729f-4423-8eac-813c01f8f72b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2923799929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2923799929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2923799929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2923799929}]}, 'public_addr': '172.21.15.99:6825/2923799929', 'cluster_addr': '172.21.15.99:6827/2923799929', 'heartbeat_back_addr': '172.21.15.99:6831/2923799929', 'heartbeat_front_addr': '172.21.15.99:6829/2923799929', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.267895+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.161313+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.447964+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.021320+0000', 'dead_epoch': 48}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.883961+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.938146+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.627237+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.887523+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:27:06.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:27:06.379 INFO:teuthology.orchestra.run.smithi063.stdout:3143 trunc from 0xe423877 to 0x8600807 2024-09-13T09:27:06.379 INFO:teuthology.orchestra.run.smithi063.stdout:3144 write 0x1cf34c7 thru 0x1cf4dad (0x18e7 bytes) 2024-09-13T09:27:06.395 INFO:teuthology.orchestra.run.smithi063.stdout:3145 write 0x8ed45d0 thru 0x8edc3e0 (0x7e11 bytes) 2024-09-13T09:27:06.402 INFO:teuthology.orchestra.run.smithi063.stdout:3146 write 0x8772cb0 thru 0x877664d (0x399e bytes) 2024-09-13T09:27:06.409 INFO:teuthology.orchestra.run.smithi063.stdout:3148 clone 28 order 24 su 1048576 sc 9 2024-09-13T09:27:06.425 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:06.424+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130926895000002, 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:06.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:06.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130911436000002, 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:06.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:06.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130895356, 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130879582999999, 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130857681999998, 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:06.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130841795999999, 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:06.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:06.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130826481, 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:06.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:06.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130810436000001, 2024-09-13T09:27:06.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:06.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130794712, 2024-09-13T09:27:06.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:06.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.529 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130780112, 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:06.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130764683999999, 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.537 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.130748797999999, 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85175 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.781635822999998, 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048714652999999997, 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85175 2024-09-13T09:27:06.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:06.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707870+0000", 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.131e-06 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707875+0000", 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9099999999999996e-06 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707879+0000", 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2690000000000001e-06 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.753863+0000", 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045983250000000003 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.754182+0000", 2024-09-13T09:27:06.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031968300000000002 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756583+0000", 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024007339999999999 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.781621555000001, 2024-09-13T09:27:06.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:06.550 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.781487188, 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.781466297000001, 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:06.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:06.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:06.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97491 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 457728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.439268954999999, 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048679549000000003, 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97491 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050237+0000", 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050243+0000", 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6740000000000002e-06 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050250+0000", 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4609999999999996e-06 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098540+0000", 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048290050000000001 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098608+0000", 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8645000000000004e-05 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098915+0000", 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030641899999999999 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.407416692, 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:06.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:06.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.406539132000001, 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.569 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:06.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.406499572, 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:06.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:06.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.578 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:06.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:06.580 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:06.576+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:06.633 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:06.680 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:06.675+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:06.690 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:06.691 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:06.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:06.717 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:06.721 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:27:06.733 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-13T09:27:06.539267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1091667613,"num_objects":1024,"num_object_clones":221,"num_object_copies":2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1024,"num_whiteouts":187,"num_read":154788,"num_read_kb":468350,"num_write":13005,"num_write_kb":291940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":37017024,"num_keys_recovered":122,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9757,"ondisk_log_size":9757,"up":77,"acting":77,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":348232,"kb_used_data":50772,"kb_used_omap":25,"kb_used_meta":297318,"kb_avail":754626488,"statfs":{"total":773094113280,"available":772737523712,"internally_reserved":0,"allocated":51990528,"data_stored":330337404,"data_compressed":23631708,"data_compressed_allocated":29065216,"data_compressed_original":314537464,"omap_allocated":25946,"internal_metadata":304454310},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2825038,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-116,"num_objects_misplaced":-76,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":123,"num_read_kb":339,"num_write":22,"num_write_kb":1579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":7362353,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001115"},"pg_stats":[{"pgid":"3.1f","version":"72'192","reported_seq":845,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.494141+0000","last_change":"2024-09-13T09:26:39.600998+0000","last_active":"2024-09-13T09:27:00.494141+0000","last_peered":"2024-09-13T09:27:00.494141+0000","last_clean":"2024-09-13T09:27:00.494141+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:27:00.494141+0000","last_undegraded":"2024-09-13T09:27:00.494141+0000","last_fullsized":"2024-09-13T09:27:00.494141+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":28825361,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":2353,"num_read_kb":12452,"num_write":219,"num_write_kb":5598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"76'386","reported_seq":4649,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-13T09:27:04.732273+0000","last_change":"2024-09-13T09:27:04.731917+0000","last_active":"2024-09-13T09:27:04.731917+0000","last_peered":"2024-09-13T09:27:04.322693+0000","last_clean":"2024-09-13T09:26:25.215708+0000","last_became_active":"2024-09-13T09:27:03.718173+0000","last_became_peered":"2024-09-13T09:27:03.718173+0000","last_unstale":"2024-09-13T09:27:04.732273+0000","last_undegraded":"2024-09-13T09:27:04.732273+0000","last_fullsized":"2024-09-13T09:27:04.732273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:52.655871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":27124276,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":2,"num_read":5453,"num_read_kb":10333,"num_write":619,"num_write_kb":4390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"72'197","reported_seq":2447,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.573395+0000","last_change":"2024-09-13T09:25:55.200905+0000","last_active":"2024-09-13T09:27:00.573395+0000","last_peered":"2024-09-13T09:27:00.573395+0000","last_clean":"2024-09-13T09:27:00.573395+0000","last_became_active":"2024-09-13T09:25:53.182525+0000","last_became_peered":"2024-09-13T09:25:53.182525+0000","last_unstale":"2024-09-13T09:27:00.573395+0000","last_undegraded":"2024-09-13T09:27:00.573395+0000","last_fullsized":"2024-09-13T09:27:00.573395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:20:56.196277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":41225271,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":2727,"num_read_kb":12500,"num_write":216,"num_write_kb":7406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"73'336","reported_seq":3377,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.733480+0000","last_change":"2024-09-13T09:25:55.200942+0000","last_active":"2024-09-13T09:27:00.733480+0000","last_peered":"2024-09-13T09:27:00.733480+0000","last_clean":"2024-09-13T09:27:00.733480+0000","last_became_active":"2024-09-13T09:25:53.182441+0000","last_became_peered":"2024-09-13T09:25:53.182441+0000","last_unstale":"2024-09-13T09:27:00.733480+0000","last_undegraded":"2024-09-13T09:27:00.733480+0000","last_fullsized":"2024-09-13T09:27:00.733480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:00.059063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":56032502,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":3773,"num_read_kb":8888,"num_write":440,"num_write_kb":9221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"72'271","reported_seq":1335,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.448509+0000","last_change":"2024-09-13T09:26:42.573701+0000","last_active":"2024-09-13T09:27:00.448509+0000","last_peered":"2024-09-13T09:27:00.448509+0000","last_clean":"2024-09-13T09:27:00.448509+0000","last_became_active":"2024-09-13T09:26:42.571745+0000","last_became_peered":"2024-09-13T09:26:42.571745+0000","last_unstale":"2024-09-13T09:27:00.448509+0000","last_undegraded":"2024-09-13T09:27:00.448509+0000","last_fullsized":"2024-09-13T09:27:00.448509+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:14.230705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8156723,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":2065,"num_read_kb":8170,"num_write":332,"num_write_kb":4592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2489204,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"70'310","reported_seq":9794,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.754218+0000","last_change":"2024-09-13T09:25:55.187594+0000","last_active":"2024-09-13T09:27:00.754218+0000","last_peered":"2024-09-13T09:27:00.754218+0000","last_clean":"2024-09-13T09:27:00.754218+0000","last_became_active":"2024-09-13T09:25:53.167913+0000","last_became_peered":"2024-09-13T09:25:53.167913+0000","last_unstale":"2024-09-13T09:27:00.754218+0000","last_undegraded":"2024-09-13T09:27:00.754218+0000","last_fullsized":"2024-09-13T09:27:00.754218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:18:31.544056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":44001026,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":5,"num_read":11194,"num_read_kb":17886,"num_write":328,"num_write_kb":4734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"72'280","reported_seq":1054,"reported_epoch":72,"state":"active+undersized+degraded","last_fresh":"2024-09-13T09:27:00.490889+0000","last_change":"2024-09-13T09:26:27.430763+0000","last_active":"2024-09-13T09:27:00.490889+0000","last_peered":"2024-09-13T09:27:00.490889+0000","last_clean":"2024-09-13T09:26:25.425749+0000","last_became_active":"2024-09-13T09:26:27.430763+0000","last_became_peered":"2024-09-13T09:26:27.430763+0000","last_unstale":"2024-09-13T09:27:00.490889+0000","last_undegraded":"2024-09-13T09:26:27.429857+0000","last_fullsized":"2024-09-13T09:26:27.429728+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:10.663498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":24822635,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":1495,"num_read_kb":7066,"num_write":290,"num_write_kb":6739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"72'230","reported_seq":793,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T09:27:03.988572+0000","last_change":"2024-09-13T09:27:03.781188+0000","last_active":"2024-09-13T09:27:03.988572+0000","last_peered":"2024-09-13T09:27:03.988572+0000","last_clean":"2024-09-13T09:27:03.988572+0000","last_became_active":"2024-09-13T09:27:03.711948+0000","last_became_peered":"2024-09-13T09:27:03.711948+0000","last_unstale":"2024-09-13T09:27:03.988572+0000","last_undegraded":"2024-09-13T09:27:03.988572+0000","last_fullsized":"2024-09-13T09:27:03.988572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:42.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":12751976,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":4,"num_read":1877,"num_read_kb":10469,"num_write":248,"num_write_kb":9945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":8502479,"num_keys_recovered":39,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"72'318","reported_seq":646,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.873147+0000","last_change":"2024-09-13T09:26:42.575210+0000","last_active":"2024-09-13T09:26:59.873147+0000","last_peered":"2024-09-13T09:26:59.873147+0000","last_clean":"2024-09-13T09:26:59.873147+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:26:59.873147+0000","last_undegraded":"2024-09-13T09:26:59.873147+0000","last_fullsized":"2024-09-13T09:26:59.873147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:38:26.807120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3417031,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":1244,"num_read_kb":2185,"num_write":399,"num_write_kb":2079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"72'226","reported_seq":4703,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.492480+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:27:00.492480+0000","last_peered":"2024-09-13T09:27:00.492480+0000","last_clean":"2024-09-13T09:27:00.492480+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:27:00.492480+0000","last_undegraded":"2024-09-13T09:27:00.492480+0000","last_fullsized":"2024-09-13T09:27:00.492480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":47637708,"num_objects":34,"num_object_clones":11,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":6,"num_read":4780,"num_read_kb":22759,"num_write":315,"num_write_kb":10948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"73'187","reported_seq":6482,"reported_epoch":73,"state":"active+undersized+degraded","last_fresh":"2024-09-13T09:27:00.757723+0000","last_change":"2024-09-13T09:26:27.431088+0000","last_active":"2024-09-13T09:27:00.757723+0000","last_peered":"2024-09-13T09:27:00.757723+0000","last_clean":"2024-09-13T09:26:25.428255+0000","last_became_active":"2024-09-13T09:26:27.431088+0000","last_became_peered":"2024-09-13T09:26:27.431088+0000","last_unstale":"2024-09-13T09:27:00.757723+0000","last_undegraded":"2024-09-13T09:26:27.429848+0000","last_fullsized":"2024-09-13T09:26:27.429698+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:41:46.640852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":17799518,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":5,"num_read":6722,"num_read_kb":18080,"num_write":208,"num_write_kb":4525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":34}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"70'157","reported_seq":3547,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.674582+0000","last_change":"2024-09-13T09:25:55.188342+0000","last_active":"2024-09-13T09:26:59.674582+0000","last_peered":"2024-09-13T09:26:59.674582+0000","last_clean":"2024-09-13T09:26:59.674582+0000","last_became_active":"2024-09-13T09:25:53.178133+0000","last_became_peered":"2024-09-13T09:25:53.178133+0000","last_unstale":"2024-09-13T09:26:59.674582+0000","last_undegraded":"2024-09-13T09:26:59.674582+0000","last_fullsized":"2024-09-13T09:26:59.674582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:01.188801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":30539707,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":6,"num_read":3367,"num_read_kb":7714,"num_write":169,"num_write_kb":2625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":72,"state":"active+undersized","last_fresh":"2024-09-13T09:26:59.775905+0000","last_change":"2024-09-13T09:26:27.439943+0000","last_active":"2024-09-13T09:26:59.775905+0000","last_peered":"2024-09-13T09:26:59.775905+0000","last_clean":"2024-09-13T09:26:25.804653+0000","last_became_active":"2024-09-13T09:26:27.439943+0000","last_became_peered":"2024-09-13T09:26:27.439943+0000","last_unstale":"2024-09-13T09:26:59.775905+0000","last_undegraded":"2024-09-13T09:26:59.775905+0000","last_fullsized":"2024-09-13T09:26:27.437517+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:15.288691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"73'198","reported_seq":715,"reported_epoch":73,"state":"active+undersized+degraded","last_fresh":"2024-09-13T09:27:00.759580+0000","last_change":"2024-09-13T09:26:27.439510+0000","last_active":"2024-09-13T09:27:00.759580+0000","last_peered":"2024-09-13T09:27:00.759580+0000","last_clean":"2024-09-13T09:26:25.804748+0000","last_became_active":"2024-09-13T09:26:27.439510+0000","last_became_peered":"2024-09-13T09:26:27.439510+0000","last_unstale":"2024-09-13T09:27:00.759580+0000","last_undegraded":"2024-09-13T09:26:27.438575+0000","last_fullsized":"2024-09-13T09:26:27.438511+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:53:59.415674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":49626300,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":5,"num_read":523,"num_read_kb":4174,"num_write":217,"num_write_kb":6149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":40}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":90,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.776064+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:26:59.776064+0000","last_peered":"2024-09-13T09:26:59.776064+0000","last_clean":"2024-09-13T09:26:59.776064+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:26:59.776064+0000","last_undegraded":"2024-09-13T09:26:59.776064+0000","last_fullsized":"2024-09-13T09:26:59.776064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:40:48.058697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"72'263","reported_seq":2301,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.449303+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:27:00.449303+0000","last_peered":"2024-09-13T09:27:00.449303+0000","last_clean":"2024-09-13T09:27:00.449303+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:27:00.449303+0000","last_undegraded":"2024-09-13T09:27:00.449303+0000","last_fullsized":"2024-09-13T09:27:00.449303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":21713553,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":3578,"num_read_kb":12359,"num_write":347,"num_write_kb":17726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.674724+0000","last_change":"2024-09-13T09:25:45.109035+0000","last_active":"2024-09-13T09:26:59.674724+0000","last_peered":"2024-09-13T09:26:59.674724+0000","last_clean":"2024-09-13T09:26:59.674724+0000","last_became_active":"2024-09-13T09:25:42.953067+0000","last_became_peered":"2024-09-13T09:25:42.953067+0000","last_unstale":"2024-09-13T09:26:59.674724+0000","last_undegraded":"2024-09-13T09:26:59.674724+0000","last_fullsized":"2024-09-13T09:26:59.674724+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:00.879788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089485499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"76'230","reported_seq":3718,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-13T09:27:04.737618+0000","last_change":"2024-09-13T09:27:04.732818+0000","last_active":"2024-09-13T09:27:04.732818+0000","last_peered":"2024-09-13T09:27:03.719509+0000","last_clean":"2024-09-13T09:26:25.002026+0000","last_became_active":"2024-09-13T09:27:03.718570+0000","last_became_peered":"2024-09-13T09:27:03.718570+0000","last_unstale":"2024-09-13T09:27:04.737618+0000","last_undegraded":"2024-09-13T09:27:04.737618+0000","last_fullsized":"2024-09-13T09:27:04.737618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:41.612795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":56184145,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":4309,"num_read_kb":18723,"num_write":230,"num_write_kb":5973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T09:27:03.704423+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:27:03.704423+0000","last_peered":"2024-09-13T09:27:03.704423+0000","last_clean":"2024-09-13T09:27:03.704423+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:27:03.704423+0000","last_undegraded":"2024-09-13T09:27:03.704423+0000","last_fullsized":"2024-09-13T09:27:03.704423+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"70'253","reported_seq":3240,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.045802+0000","last_change":"2024-09-13T09:26:42.570158+0000","last_active":"2024-09-13T09:27:00.045802+0000","last_peered":"2024-09-13T09:27:00.045802+0000","last_clean":"2024-09-13T09:27:00.045802+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:27:00.045802+0000","last_undegraded":"2024-09-13T09:27:00.045802+0000","last_fullsized":"2024-09-13T09:27:00.045802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41268938,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":8,"num_read":4059,"num_read_kb":15876,"num_write":254,"num_write_kb":3993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"70'562","reported_seq":1542,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.676917+0000","last_change":"2024-09-13T09:26:43.572479+0000","last_active":"2024-09-13T09:26:59.676917+0000","last_peered":"2024-09-13T09:26:59.676917+0000","last_clean":"2024-09-13T09:26:59.676917+0000","last_became_active":"2024-09-13T09:26:43.571008+0000","last_became_peered":"2024-09-13T09:26:43.571008+0000","last_unstale":"2024-09-13T09:26:59.676917+0000","last_undegraded":"2024-09-13T09:26:59.676917+0000","last_fullsized":"2024-09-13T09:26:59.676917+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:23.127176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10483188,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":3,"num_read":4050,"num_read_kb":12606,"num_write":880,"num_write_kb":5002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":7377,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":213,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.686639+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:26:59.686639+0000","last_peered":"2024-09-13T09:26:59.686639+0000","last_clean":"2024-09-13T09:26:59.686639+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:26:59.686639+0000","last_undegraded":"2024-09-13T09:26:59.686639+0000","last_fullsized":"2024-09-13T09:26:59.686639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":71,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.676046+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:26:59.676046+0000","last_peered":"2024-09-13T09:26:59.676046+0000","last_clean":"2024-09-13T09:26:59.676046+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:26:59.676046+0000","last_undegraded":"2024-09-13T09:26:59.676046+0000","last_fullsized":"2024-09-13T09:26:59.676046+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"72'528","reported_seq":7280,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.843361+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:26:59.843361+0000","last_peered":"2024-09-13T09:26:59.843361+0000","last_clean":"2024-09-13T09:26:59.843361+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:26:59.843361+0000","last_undegraded":"2024-09-13T09:26:59.843361+0000","last_fullsized":"2024-09-13T09:26:59.843361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":36168728,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":4,"num_read":9107,"num_read_kb":11679,"num_write":821,"num_write_kb":12216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":95,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.686801+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:26:59.686801+0000","last_peered":"2024-09-13T09:26:59.686801+0000","last_clean":"2024-09-13T09:26:59.686801+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:26:59.686801+0000","last_undegraded":"2024-09-13T09:26:59.686801+0000","last_fullsized":"2024-09-13T09:26:59.686801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"72'302","reported_seq":9657,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.492202+0000","last_change":"2024-09-13T09:26:42.574377+0000","last_active":"2024-09-13T09:27:00.492202+0000","last_peered":"2024-09-13T09:27:00.492202+0000","last_clean":"2024-09-13T09:27:00.492202+0000","last_became_active":"2024-09-13T09:26:42.573616+0000","last_became_peered":"2024-09-13T09:26:42.573616+0000","last_unstale":"2024-09-13T09:27:00.492202+0000","last_undegraded":"2024-09-13T09:27:00.492202+0000","last_fullsized":"2024-09-13T09:27:00.492202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:49.284041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":62995623,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":8,"num_read":9661,"num_read_kb":34659,"num_write":405,"num_write_kb":27219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":10577864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":86,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.682800+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:26:59.682800+0000","last_peered":"2024-09-13T09:26:59.682800+0000","last_clean":"2024-09-13T09:26:59.682800+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:26:59.682800+0000","last_undegraded":"2024-09-13T09:26:59.682800+0000","last_fullsized":"2024-09-13T09:26:59.682800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"76'167","reported_seq":4097,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-13T09:27:04.736150+0000","last_change":"2024-09-13T09:27:04.731846+0000","last_active":"2024-09-13T09:27:04.731846+0000","last_peered":"2024-09-13T09:27:04.070444+0000","last_clean":"2024-09-13T09:26:25.161184+0000","last_became_active":"2024-09-13T09:27:03.716285+0000","last_became_peered":"2024-09-13T09:27:03.716285+0000","last_unstale":"2024-09-13T09:27:04.736150+0000","last_undegraded":"2024-09-13T09:27:04.736150+0000","last_fullsized":"2024-09-13T09:27:04.736150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:47.672205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":18369107,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":3919,"num_read_kb":4139,"num_write":175,"num_write_kb":2515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.686617+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:26:59.686617+0000","last_peered":"2024-09-13T09:26:59.686617+0000","last_clean":"2024-09-13T09:26:59.686617+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:26:59.686617+0000","last_undegraded":"2024-09-13T09:26:59.686617+0000","last_fullsized":"2024-09-13T09:26:59.686617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"76'273","reported_seq":1161,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-13T09:27:04.732480+0000","last_change":"2024-09-13T09:27:04.731900+0000","last_active":"2024-09-13T09:27:04.731900+0000","last_peered":"2024-09-13T09:27:04.268555+0000","last_clean":"2024-09-13T09:26:25.160437+0000","last_became_active":"2024-09-13T09:27:03.713855+0000","last_became_peered":"2024-09-13T09:27:03.713855+0000","last_unstale":"2024-09-13T09:27:04.732480+0000","last_undegraded":"2024-09-13T09:27:04.732480+0000","last_fullsized":"2024-09-13T09:27:04.732480+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:11:23.383467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":17163604,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":6,"num_read":2013,"num_read_kb":8158,"num_write":334,"num_write_kb":5985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":723838,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"72'186","reported_seq":9518,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.492063+0000","last_change":"2024-09-13T09:25:55.187602+0000","last_active":"2024-09-13T09:27:00.492063+0000","last_peered":"2024-09-13T09:27:00.492063+0000","last_clean":"2024-09-13T09:27:00.492063+0000","last_became_active":"2024-09-13T09:25:53.646396+0000","last_became_peered":"2024-09-13T09:25:53.646396+0000","last_unstale":"2024-09-13T09:27:00.492063+0000","last_undegraded":"2024-09-13T09:27:00.492063+0000","last_fullsized":"2024-09-13T09:27:00.492063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:17:21.102946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":37282091,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":9327,"num_read_kb":16418,"num_write":183,"num_write_kb":10555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"72'151","reported_seq":1676,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.778483+0000","last_change":"2024-09-13T09:25:55.752827+0000","last_active":"2024-09-13T09:26:59.778483+0000","last_peered":"2024-09-13T09:26:59.778483+0000","last_clean":"2024-09-13T09:26:59.778483+0000","last_became_active":"2024-09-13T09:25:53.180652+0000","last_became_peered":"2024-09-13T09:25:53.180652+0000","last_unstale":"2024-09-13T09:26:59.778483+0000","last_undegraded":"2024-09-13T09:26:59.778483+0000","last_fullsized":"2024-09-13T09:26:59.778483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:31.395047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":15395722,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1521,"num_read_kb":8875,"num_write":151,"num_write_kb":8838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"76'269","reported_seq":3023,"reported_epoch":77,"state":"peering","last_fresh":"2024-09-13T09:27:04.733142+0000","last_change":"2024-09-13T09:27:04.731263+0000","last_active":"2024-09-13T09:27:04.731263+0000","last_peered":"2024-09-13T09:27:04.313672+0000","last_clean":"2024-09-13T09:27:00.717566+0000","last_became_active":"2024-09-13T09:27:03.711758+0000","last_became_peered":"2024-09-13T09:27:03.711758+0000","last_unstale":"2024-09-13T09:27:04.733142+0000","last_undegraded":"2024-09-13T09:27:04.733142+0000","last_fullsized":"2024-09-13T09:27:04.733142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:57.112483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":63853536,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":9,"num_read":3005,"num_read_kb":21288,"num_write":348,"num_write_kb":18392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"72'387","reported_seq":5940,"reported_epoch":72,"state":"active+undersized+degraded","last_fresh":"2024-09-13T09:27:00.449293+0000","last_change":"2024-09-13T09:26:27.439782+0000","last_active":"2024-09-13T09:27:00.449293+0000","last_peered":"2024-09-13T09:27:00.449293+0000","last_clean":"2024-09-13T09:26:25.804248+0000","last_became_active":"2024-09-13T09:26:27.439782+0000","last_became_peered":"2024-09-13T09:26:27.439782+0000","last_unstale":"2024-09-13T09:27:00.449293+0000","last_undegraded":"2024-09-13T09:26:27.438055+0000","last_fullsized":"2024-09-13T09:26:27.437946+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:49:18.776534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":39529719,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":6632,"num_read_kb":20647,"num_write":600,"num_write_kb":14636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":26}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"73'211","reported_seq":12805,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.787283+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:27:00.787283+0000","last_peered":"2024-09-13T09:27:00.787283+0000","last_clean":"2024-09-13T09:27:00.787283+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:27:00.787283+0000","last_undegraded":"2024-09-13T09:27:00.787283+0000","last_fullsized":"2024-09-13T09:27:00.787283+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":65364850,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":12625,"num_read_kb":29916,"num_write":248,"num_write_kb":27322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"73'887","reported_seq":6769,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.789592+0000","last_change":"2024-09-13T09:25:55.220725+0000","last_active":"2024-09-13T09:27:00.789592+0000","last_peered":"2024-09-13T09:27:00.789592+0000","last_clean":"2024-09-13T09:27:00.789592+0000","last_became_active":"2024-09-13T09:25:53.178164+0000","last_became_peered":"2024-09-13T09:25:53.178164+0000","last_unstale":"2024-09-13T09:27:00.789592+0000","last_undegraded":"2024-09-13T09:27:00.789592+0000","last_fullsized":"2024-09-13T09:27:00.789592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:07:52.167054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":38304169,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":9,"num_read":9958,"num_read_kb":17543,"num_write":1521,"num_write_kb":13237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"72'204","reported_seq":5074,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.492040+0000","last_change":"2024-09-13T09:25:55.219599+0000","last_active":"2024-09-13T09:27:00.492040+0000","last_peered":"2024-09-13T09:27:00.492040+0000","last_clean":"2024-09-13T09:27:00.492040+0000","last_became_active":"2024-09-13T09:25:53.177574+0000","last_became_peered":"2024-09-13T09:25:53.177574+0000","last_unstale":"2024-09-13T09:27:00.492040+0000","last_undegraded":"2024-09-13T09:27:00.492040+0000","last_fullsized":"2024-09-13T09:27:00.492040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:30:35.248811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":46423155,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":4976,"num_read_kb":27918,"num_write":240,"num_write_kb":11639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"72'391","reported_seq":1672,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.449249+0000","last_change":"2024-09-13T09:26:42.574072+0000","last_active":"2024-09-13T09:27:00.449249+0000","last_peered":"2024-09-13T09:27:00.449249+0000","last_clean":"2024-09-13T09:27:00.449249+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:27:00.449249+0000","last_undegraded":"2024-09-13T09:27:00.449249+0000","last_fullsized":"2024-09-13T09:27:00.449249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":33234935,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":9,"num_read":2405,"num_read_kb":17345,"num_write":611,"num_write_kb":6938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"70'409","reported_seq":4559,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:26:59.901545+0000","last_change":"2024-09-13T09:25:55.219871+0000","last_active":"2024-09-13T09:26:59.901545+0000","last_peered":"2024-09-13T09:26:59.901545+0000","last_clean":"2024-09-13T09:26:59.901545+0000","last_became_active":"2024-09-13T09:25:53.179182+0000","last_became_peered":"2024-09-13T09:25:53.179182+0000","last_unstale":"2024-09-13T09:26:59.901545+0000","last_undegraded":"2024-09-13T09:26:59.901545+0000","last_fullsized":"2024-09-13T09:26:59.901545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:45.198757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":31631496,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":5514,"num_read_kb":15687,"num_write":595,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"72'189","reported_seq":4070,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.042871+0000","last_change":"2024-09-13T09:26:42.576092+0000","last_active":"2024-09-13T09:27:00.042871+0000","last_peered":"2024-09-13T09:27:00.042871+0000","last_clean":"2024-09-13T09:27:00.042871+0000","last_became_active":"2024-09-13T09:26:42.575623+0000","last_became_peered":"2024-09-13T09:26:42.575623+0000","last_unstale":"2024-09-13T09:27:00.042871+0000","last_undegraded":"2024-09-13T09:27:00.042871+0000","last_fullsized":"2024-09-13T09:27:00.042871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:22:21.827879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32582502,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":3760,"num_read_kb":16346,"num_write":245,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2521446,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"72'487","reported_seq":5171,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-13T09:27:00.449839+0000","last_change":"2024-09-13T09:25:55.219148+0000","last_active":"2024-09-13T09:27:00.449839+0000","last_peered":"2024-09-13T09:27:00.449839+0000","last_clean":"2024-09-13T09:27:00.449839+0000","last_became_active":"2024-09-13T09:25:53.177434+0000","last_became_peered":"2024-09-13T09:25:53.177434+0000","last_unstale":"2024-09-13T09:27:00.449839+0000","last_undegraded":"2024-09-13T09:27:00.449839+0000","last_fullsized":"2024-09-13T09:27:00.449839+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:13:59.794157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":31168131,"num_objects":43,"num_object_clones":13,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":6680,"num_read_kb":15392,"num_write":656,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1091077226,"num_objects":1020,"num_object_clones":221,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":187,"num_read":154672,"num_read_kb":468250,"num_write":12844,"num_write_kb":289522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":474,"num_bytes_recovered":37016986,"num_keys_recovered":122,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":44990464,"data_stored":326038129,"data_compressed":23565694,"data_compressed_allocated":28975104,"data_compressed_original":313390584,"omap_allocated":8553,"internal_metadata":0},"log_size":9637,"ondisk_log_size":9637,"up":60,"acting":60,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":15,"acting":15,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47448,"kb_used_data":8140,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324392,"statfs":{"total":96636764160,"available":96588177408,"internally_reserved":0,"allocated":8335360,"data_stored":55083946,"data_compressed":3859991,"data_compressed_allocated":4304896,"data_compressed_original":51934643,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.995},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.63200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.745},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":8.2249999999999996,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.86699999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.087}]}]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33644,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338196,"statfs":{"total":96636764160,"available":96602312704,"internally_reserved":0,"allocated":2584576,"data_stored":9581304,"data_compressed":775818,"data_compressed_allocated":1032192,"data_compressed_original":8686246,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.498},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.0339999999999998,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.149,"5min":5.149,"15min":5.149},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.88100000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":5.0709999999999997,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.98899999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.087},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":6.2050000000000001,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.014}]}]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47404,"kb_used_data":8092,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324436,"statfs":{"total":96636764160,"available":96588222464,"internally_reserved":0,"allocated":8286208,"data_stored":51471256,"data_compressed":3831978,"data_compressed_allocated":4702208,"data_compressed_original":49007428,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.92100000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.629}]},{"osd":2,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.0029999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.95699999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.98199999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54476,"kb_used_data":9400,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94317364,"statfs":{"total":96636764160,"available":96580980736,"internally_reserved":0,"allocated":9625600,"data_stored":64945436,"data_compressed":4486468,"data_compressed_allocated":5353472,"data_compressed_original":61680915,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.54900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.78700000000000003}]}]},{"osd":3,"up_from":74,"seq":317827579906,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40144,"kb_used_data":7232,"kb_used_omap":14,"kb_used_meta":32881,"kb_avail":94331696,"statfs":{"total":96636764160,"available":96595656704,"internally_reserved":0,"allocated":7405568,"data_stored":50758569,"data_compressed":3259187,"data_compressed_allocated":4337664,"data_compressed_original":48689152,"omap_allocated":14794,"internal_metadata":33670710},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43836,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328004,"statfs":{"total":96636764160,"available":96591876096,"internally_reserved":0,"allocated":4636672,"data_stored":29198123,"data_compressed":2052302,"data_compressed_allocated":2727936,"data_compressed_original":28031837,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.496}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.753}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.876},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.81699999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.95399999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43276,"kb_used_data":8064,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94328564,"statfs":{"total":96636764160,"available":96592449536,"internally_reserved":0,"allocated":8257536,"data_stored":55598265,"data_compressed":4220793,"data_compressed_allocated":5181440,"data_compressed_original":53656289,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.495},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.016},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.042},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.1259999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.095},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.0760000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38004,"kb_used_data":2792,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333836,"statfs":{"total":96636764160,"available":96597848064,"internally_reserved":0,"allocated":2859008,"data_stored":13700505,"data_compressed":1145171,"data_compressed_allocated":1425408,"data_compressed_original":12850954,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":1.2869999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.018}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.173},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.335}]},{"osd":5,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.0569999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.1040000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7405568,"data_stored":55213740,"data_compressed":4220793,"data_compressed_allocated":5181440,"data_compressed_original":53656289,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3723264,"data_stored":28223249,"data_compressed":2019295,"data_compressed_allocated":2682880,"data_compressed_original":27458397,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1961984,"data_stored":13290436,"data_compressed":1145171,"data_compressed_allocated":1425408,"data_compressed_original":12850954,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6512640,"data_stored":50349436,"data_compressed":3259187,"data_compressed_allocated":4337664,"data_compressed_original":48689152,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8777728,"data_stored":64560930,"data_compressed":4486468,"data_compressed_allocated":5353472,"data_compressed_original":61680915,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7442432,"data_stored":51087686,"data_compressed":3831978,"data_compressed_allocated":4702208,"data_compressed_original":49007428,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1748992,"data_stored":9203599,"data_compressed":775818,"data_compressed_allocated":1032192,"data_compressed_original":8686246,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7417856,"data_stored":54109053,"data_compressed":3826984,"data_compressed_allocated":4259840,"data_compressed_original":51361203,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:27:06.737 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [5, 6] 2024-09-13T09:27:06.737 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '5', '6'] 2024-09-13T09:27:06.737 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 5 6 2024-09-13T09:27:06.753 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:27:06.781 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:06.775+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:06.783 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:06.807 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:06.827 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:06.853 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.853 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:06.853 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:06.853 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:06.853 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:06.854 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:06.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:06.882 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:06.875+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:06.890 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:06.972 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 67.410548953000003, 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.600325690999998, 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.341582819999999, 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.339380481999999, 2024-09-13T09:27:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.338983881000001, 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.338108497, 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.338087514000001, 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.338037002, 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.338007142999999, 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.287667461000002, 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.287608883000001, 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.287586248, 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:07.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:07.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.287569286, 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:07.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:07.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.286697556, 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:07.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:07.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:07.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.286664922, 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.286644523, 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:07.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.286602838, 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:07.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:07.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.286470176999998, 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:07.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.284119724, 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:07.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:07.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:07.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.283935729, 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:07.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:07.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:07.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:07.066 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:07.067 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T09:27:07.068 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:07.068 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:07.059+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:07.128 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:07.164 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:07.168 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:07.159+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:07.184 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:07.184 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:07.184 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:07.184 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:07.189 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:07.191 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:07.194 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:07.256 INFO:teuthology.orchestra.run.smithi063.stdout:3149 write 0xbbb1e0a thru 0xbbc08b5 (0xeaac bytes) 2024-09-13T09:27:07.269 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:07.267+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:07.277 INFO:teuthology.orchestra.run.smithi063.stdout:3150 write 0xe70af7b thru 0xe71a3c8 (0xf44e bytes) 2024-09-13T09:27:07.285 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.305 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:07.305 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:07.306 INFO:teuthology.orchestra.run.smithi063.stdout:3153 trunc from 0xe71a3c9 to 0x3f5d5c8 2024-09-13T09:27:07.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:07.361 INFO:teuthology.orchestra.run.smithi063.stdout:3154 write 0x67ecf70 thru 0x67f8f7d (0xc00e bytes) 2024-09-13T09:27:07.364 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.364 INFO:teuthology.orchestra.run.smithi063.stdout:3156 read 0x5b72507 thru 0x5b732b3 (0xdad bytes) 2024-09-13T09:27:07.365 INFO:teuthology.orchestra.run.smithi063.stdout:3158 read 0x580f5dc thru 0x5810351 (0xd76 bytes) 2024-09-13T09:27:07.370 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:07.367+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:07.375 INFO:teuthology.orchestra.run.smithi063.stdout:3159 trunc from 0x67f8f7e to 0x2731319 2024-09-13T09:27:07.412 INFO:teuthology.orchestra.run.smithi063.stdout:3160 trunc from 0x2731319 to 0xce6cd19 2024-09-13T09:27:07.418 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:07.418 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:07.422 INFO:teuthology.orchestra.run.smithi063.stdout:3161 write 0x5ec628e thru 0x5ecbe6b (0x5bde bytes) 2024-09-13T09:27:07.425 INFO:teuthology.orchestra.run.smithi063.stdout:3162 writesame 0x47dcc90 thru 0x47e3946 data_size 0x1(0x6cb7 bytes) 2024-09-13T09:27:07.429 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:07.445 INFO:teuthology.orchestra.run.smithi063.stdout:3163 read 0x7204847 thru 0x720f643 (0xadfd bytes) 2024-09-13T09:27:07.445 INFO:teuthology.orchestra.run.smithi063.stdout:3165 punch from 0x88da909 to 0x88de6e4, (0x3ddb bytes) 2024-09-13T09:27:07.455 INFO:teuthology.orchestra.run.smithi063.stdout:3166 trunc from 0xce6cd19 to 0x8f797c9 2024-09-13T09:27:07.467 INFO:teuthology.orchestra.run.smithi063.stdout:3167 write 0x1d5de12 thru 0x1d65bba (0x7da9 bytes) 2024-09-13T09:27:07.470 INFO:teuthology.orchestra.run.smithi063.stdout:3168 write 0x39993e thru 0x3a15b2 (0x7c75 bytes) 2024-09-13T09:27:07.470 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:07.467+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:07.484 INFO:teuthology.orchestra.run.smithi063.stdout:3169 trunc from 0x8f797c9 to 0xd87ce76 2024-09-13T09:27:07.494 INFO:teuthology.orchestra.run.smithi063.stdout:3170 punch from 0x8f624f8 to 0x8f6ddc8, (0xb8d0 bytes) 2024-09-13T09:27:07.494 INFO:teuthology.orchestra.run.smithi063.stdout:3172 read 0x1abc746 thru 0x1ac46a2 (0x7f5d bytes) 2024-09-13T09:27:07.505 INFO:teuthology.orchestra.run.smithi063.stdout:3173 trunc from 0xd87ce76 to 0x7d8a224 2024-09-13T09:27:07.535 INFO:teuthology.orchestra.run.smithi063.stdout:3174 trunc from 0x7d8a224 to 0xd552ca6 2024-09-13T09:27:07.544 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.555 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:07.555 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:07.560 INFO:teuthology.orchestra.run.smithi063.stdout:3176 trunc from 0xd552ca6 to 0x2e34848 2024-09-13T09:27:07.568 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:07.571 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:07.567+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:07.580 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.600 INFO:teuthology.orchestra.run.smithi063.stdout:3179 write 0x9eb7d10 thru 0x9ec10b4 (0x93a5 bytes) 2024-09-13T09:27:07.606 INFO:teuthology.orchestra.run.smithi063.stdout:3180 read 0x780d735 thru 0x781ccf6 (0xf5c2 bytes) 2024-09-13T09:27:07.612 INFO:teuthology.orchestra.run.smithi063.stdout:3184 trunc from 0x9ec10b5 to 0x83025d5 2024-09-13T09:27:07.640 INFO:teuthology.orchestra.run.smithi063.stdout:3186 read 0xc95a05 thru 0xc97cec (0x22e8 bytes) 2024-09-13T09:27:07.641 INFO:teuthology.orchestra.run.smithi063.stdout:3187 read 0x5a8d65e thru 0x5a95143 (0x7ae6 bytes) 2024-09-13T09:27:07.642 INFO:teuthology.orchestra.run.smithi063.stdout:3188 writesame 0x78c022f thru 0x78cef97 data_size 0x9(0xed69 bytes) 2024-09-13T09:27:07.672 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:07.667+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:07.673 INFO:teuthology.orchestra.run.smithi063.stdout:3189 write 0xdf68f29 thru 0xdf6c131 (0x3209 bytes) 2024-09-13T09:27:07.681 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:07.681 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:3190 trunc from 0xdf6c132 to 0x3c8e732 2024-09-13T09:27:07.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:07.720 INFO:teuthology.orchestra.run.smithi063.stdout:3192 trunc from 0x3c8e732 to 0x44fcc18 2024-09-13T09:27:07.728 INFO:teuthology.orchestra.run.smithi063.stdout:3193 read 0x14b078e thru 0x14bb3ac (0xac1f bytes) 2024-09-13T09:27:07.752 INFO:teuthology.orchestra.run.smithi063.stdout:3194 write 0xbcef550 thru 0xbcff161 (0xfc12 bytes) 2024-09-13T09:27:07.760 INFO:teuthology.orchestra.run.smithi063.stdout:3196 punch from 0x3dbdacb to 0x3dc8bf0, (0xb125 bytes) 2024-09-13T09:27:07.763 INFO:teuthology.orchestra.run.smithi063.stdout:3197 write 0x5a7d02e thru 0x5a870fa (0xa0cd bytes) 2024-09-13T09:27:07.765 INFO:teuthology.orchestra.run.smithi063.stderr:set 3.12 pg_upmap mapping to [5,6] 2024-09-13T09:27:07.765 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:27:07.772 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:07.767+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:07.773 INFO:teuthology.orchestra.run.smithi063.stdout:3198 trunc from 0xbcff162 to 0xa219c5f 2024-09-13T09:27:07.784 INFO:teuthology.orchestra.run.smithi063.stdout:3200 read 0x739d078 thru 0x73ab319 (0xe2a2 bytes) 2024-09-13T09:27:07.800 INFO:teuthology.orchestra.run.smithi063.stdout:3201 write 0xd9fd3e3 thru 0xda01258 (0x3e76 bytes) 2024-09-13T09:27:07.807 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.807 INFO:teuthology.orchestra.run.smithi063.stdout:3203 read 0xc7ce4f6 thru 0xc7d9cf2 (0xb7fd bytes) 2024-09-13T09:27:07.807 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.807 INFO:teuthology.orchestra.run.smithi063.stdout:3206 write 0x82d07fb thru 0x82dedca (0xe5d0 bytes) 2024-09-13T09:27:07.810 INFO:teuthology.orchestra.run.smithi063.stdout:3214 read 0x22bfb3f thru 0x22c5211 (0x56d3 bytes) 2024-09-13T09:27:07.811 INFO:teuthology.orchestra.run.smithi063.stdout:3216 write 0x445ebec thru 0x44630b2 (0x44c7 bytes) 2024-09-13T09:27:07.814 INFO:teuthology.orchestra.run.smithi063.stdout:3217 write 0xadf29f5 thru 0xadfaf22 (0x852e bytes) 2024-09-13T09:27:07.816 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:07.816 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:07.817 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:07.817 INFO:teuthology.orchestra.run.smithi063.stdout:3219 punch from 0x37c794d to 0x37c97f8, (0x1eab bytes) 2024-09-13T09:27:07.817 INFO:teuthology.orchestra.run.smithi063.stdout:3221 writesame 0xbf8d4e thru 0xc0736e data_size 0x1(0xe621 bytes) 2024-09-13T09:27:07.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:07.873 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:07.871+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:07.930 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:07.930 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:07.940 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:07.974 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:07.967+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:08.047 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.047 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.057 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:08.075 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:08.071+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:08.176 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:08.167+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:08.212 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.212 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.221 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:08.277 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:08.271+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:08.329 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.329 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.339 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:08.378 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:08.375+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:08.479 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:08.471+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:08.494 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.494 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.504 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:08.581 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:08.575+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:08.611 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.611 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:08.681 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:08.675+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:08.734 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.734 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.744 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:08.782 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:08.775+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:08.850 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.850 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.859 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:08.909 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:08.903+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:08.965 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:08.965 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:08.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:08.984 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:08.975+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:09.085 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:09.079+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:09.095 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:09.095 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:09.105 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:09.186 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:09.183+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:09.202 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:09.214 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:09.214 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:09.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:09.287 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:09.279+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:09.357 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:09.375 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:09.375 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:09.380 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:09.384 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:09.387 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:09.379+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:09.404 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:09.426 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:09.449 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:09.473 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:09.488 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:09.479+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:09.490 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:09.490 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:09.495 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:09.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:09.518 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:09.521 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:09.521 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:09.521 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:09.521 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:09.521 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:09.521 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:09.522 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:09.522 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:09.589 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:09.587+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:09.609 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:09.609 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:09.609 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:09.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:09.690 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:09.687+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:09.721 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:09.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360787177999999, 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:09.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:09.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:09.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360771718999999, 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:09.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360755639000001, 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:09.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360739865999999, 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:09.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360717964999999, 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360702078999999, 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:09.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:09.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:09.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360686764, 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:09.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:09.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:09.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360670719000002, 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:09.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360654995000001, 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:09.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360640395000001, 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:09.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360624967, 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.360609081, 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:09.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:09.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85175 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1007616~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.011496105999999, 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048714652999999997, 2024-09-13T09:27:09.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85175 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:09.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707868+0000", 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707870+0000", 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.131e-06 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707875+0000", 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9099999999999996e-06 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707879+0000", 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2690000000000001e-06 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.753863+0000", 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045983250000000003 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.754182+0000", 2024-09-13T09:27:09.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031968300000000002 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756583+0000", 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024007339999999999 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:09.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.011481838000002, 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:09.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:09.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:09.793 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.011347471000001, 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:09.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.011326579999999, 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97491 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 457728~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:09.804 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.669129238, 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048679549000000003, 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97491 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050235+0000", 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050237+0000", 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6809999999999998e-06 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050243+0000", 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6740000000000002e-06 2024-09-13T09:27:09.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.050250+0000", 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4609999999999996e-06 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098540+0000", 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048290050000000001 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098608+0000", 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8645000000000004e-05 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.098915+0000", 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030641899999999999 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.637276974999999, 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:09.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:09.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:09.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:09.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.636399415, 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:09.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.636359854999998, 2024-09-13T09:27:09.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:09.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:09.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:09.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:09.825 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:09.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:27:09.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:09.828 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:09.823+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:09.929 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:09.923+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:09.981 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:09.981 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:09.981 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:09.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:10.029 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:10.023+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:10.042 INFO:teuthology.orchestra.run.smithi063.stdout:3222 write 0x8cbeaba thru 0x8cbf79b (0xce2 bytes) 2024-09-13T09:27:10.049 INFO:teuthology.orchestra.run.smithi063.stdout:3226 trunc from 0xda01259 to 0xdb202ea 2024-09-13T09:27:10.058 INFO:teuthology.orchestra.run.smithi063.stdout:3227 write 0x4df0300 thru 0x4e00084 (0xfd85 bytes) 2024-09-13T09:27:10.061 INFO:teuthology.orchestra.run.smithi063.stdout:3229 write 0x35b7c76 thru 0x35c5a24 (0xddaf bytes) 2024-09-13T09:27:10.064 INFO:teuthology.orchestra.run.smithi063.stdout:3230 read 0xa53ed26 thru 0xa54ab34 (0xbe0f bytes) 2024-09-13T09:27:10.064 INFO:teuthology.orchestra.run.smithi063.stdout:3231 write 0x9418595 thru 0x941a713 (0x217f bytes) 2024-09-13T09:27:10.069 INFO:teuthology.orchestra.run.smithi063.stdout:3232 read 0xc140735 thru 0xc148339 (0x7c05 bytes) 2024-09-13T09:27:10.069 INFO:teuthology.orchestra.run.smithi063.stdout:3233 read 0xb730788 thru 0xb73d180 (0xc9f9 bytes) 2024-09-13T09:27:10.069 INFO:teuthology.orchestra.run.smithi063.stdout:3235 read 0x45ad8b4 thru 0x45af6b0 (0x1dfd bytes) 2024-09-13T09:27:10.070 INFO:teuthology.orchestra.run.smithi063.stdout:3236 read 0xdb1cf3b thru 0xdb202e9 (0x33af bytes) 2024-09-13T09:27:10.070 INFO:teuthology.orchestra.run.smithi063.stdout:3237 read 0x751193f thru 0x75160d2 (0x4794 bytes) 2024-09-13T09:27:10.071 INFO:teuthology.orchestra.run.smithi063.stdout:3240 write 0x51b7abb thru 0x51c59d5 (0xdf1b bytes) 2024-09-13T09:27:10.083 INFO:teuthology.orchestra.run.smithi063.stdout:3242 trunc from 0xdb202ea to 0x8826e7f 2024-09-13T09:27:10.105 INFO:teuthology.orchestra.run.smithi063.stdout:3246 punch from 0x3693f94 to 0x369db92, (0x9bfe bytes) 2024-09-13T09:27:10.105 INFO:teuthology.orchestra.run.smithi063.stdout:3247 writesame 0x2e4bcda thru 0x2e4edb1 data_size 0xc(0x30d8 bytes) 2024-09-13T09:27:10.107 INFO:teuthology.orchestra.run.smithi063.stdout:3248 write 0x596c95a thru 0x59719bc (0x5063 bytes) 2024-09-13T09:27:10.109 INFO:teuthology.orchestra.run.smithi063.stdout:3249 read 0x42c419d thru 0x42c6d94 (0x2bf8 bytes) 2024-09-13T09:27:10.109 INFO:teuthology.orchestra.run.smithi063.stdout:3250 writesame 0x521cc01 thru 0x521d907 data_size 0x17(0xd07 bytes) 2024-09-13T09:27:10.112 INFO:teuthology.orchestra.run.smithi063.stdout:3252 punch from 0x1e4f9f0 to 0x1e5bb92, (0xc1a2 bytes) 2024-09-13T09:27:10.112 INFO:teuthology.orchestra.run.smithi063.stdout:3253 trunc from 0x8826e7f to 0x8238db7 2024-09-13T09:27:10.116 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:10.116 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:10.116 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:10.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:10.130 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:10.123+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:10.130 INFO:teuthology.orchestra.run.smithi063.stdout:3254 writesame 0x44a2cc1 thru 0x44a302f data_size 0x1(0x36f bytes) 2024-09-13T09:27:10.148 INFO:teuthology.orchestra.run.smithi063.stdout:3255 write 0xc5b388e thru 0xc5be17f (0xa8f2 bytes) 2024-09-13T09:27:10.153 INFO:teuthology.orchestra.run.smithi063.stdout:3256 trunc from 0xc5be180 to 0xc1659eb 2024-09-13T09:27:10.165 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:10.165 INFO:teuthology.orchestra.run.smithi063.stdout:3261 write 0x2a049a6 thru 0x2a0a0d1 (0x572c bytes) 2024-09-13T09:27:10.168 INFO:teuthology.orchestra.run.smithi063.stdout:3262 read 0x99e0d82 thru 0x99ebf15 (0xb194 bytes) 2024-09-13T09:27:10.184 INFO:teuthology.orchestra.run.smithi063.stdout:3264 trunc from 0xc1659eb to 0x3cba5d8 2024-09-13T09:27:10.205 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:10.205 INFO:teuthology.orchestra.run.smithi063.stdout:3267 read 0x16a7b7b thru 0x16b1374 (0x97fa bytes) 2024-09-13T09:27:10.224 INFO:teuthology.orchestra.run.smithi063.stdout:3269 trunc from 0x3cba5d8 to 0xe9f9385 2024-09-13T09:27:10.231 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:10.227+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:10.233 INFO:teuthology.orchestra.run.smithi063.stdout:3270 write 0x3dc9e81 thru 0x3dd7fde (0xe15e bytes) 2024-09-13T09:27:10.236 INFO:teuthology.orchestra.run.smithi063.stdout:3271 read 0xa9371e7 thru 0xa9402dd (0x90f7 bytes) 2024-09-13T09:27:10.236 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:10.236 INFO:teuthology.orchestra.run.smithi063.stdout:3273 writesame 0x640314 thru 0x649299 data_size 0x1(0x8f86 bytes) 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.682024042999998, 2024-09-13T09:27:10.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:10.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.871800781000001, 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:10.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.613057909999998, 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:10.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.260 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.610855571999998, 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:10.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:10.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.610458971, 2024-09-13T09:27:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:10.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.609583586999999, 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:10.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:10.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.609562604000001, 2024-09-13T09:27:10.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:10.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.609512091999999, 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:10.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.283 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.609482233000001, 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:10.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.559142551000001, 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:10.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.559083973, 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:10.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.559061337999999, 2024-09-13T09:27:10.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:10.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.559044375999999, 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:10.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:10.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.558172645999999, 2024-09-13T09:27:10.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:10.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:10.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.558140011999999, 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:10.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:10.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.558119612999999, 2024-09-13T09:27:10.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:10.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:10.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.558077927999999, 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:10.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.557945267000001, 2024-09-13T09:27:10.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:10.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:10.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.555594813999999, 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:10.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:10.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.555410818999999, 2024-09-13T09:27:10.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:10.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.338 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:10.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:10.340 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:10.335+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:10.341 INFO:teuthology.orchestra.run.smithi063.stdout:3274 trunc from 0xe9f9385 to 0xadf0cc5 2024-09-13T09:27:10.341 INFO:teuthology.orchestra.run.smithi063.stdout:3275 writesame 0xc301210 thru 0xc30b739 data_size 0x57(0xa52a bytes) 2024-09-13T09:27:10.341 INFO:teuthology.orchestra.run.smithi063.stdout:3276 writesame 0x6367b26 thru 0x6372d0d data_size 0x4(0xb1e8 bytes) 2024-09-13T09:27:10.341 INFO:teuthology.orchestra.run.smithi063.stdout:3277 write 0x14fbb5 thru 0x15c7ee (0xcc3a bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3278 punch from 0xd8bb0d to 0xd8e104, (0x25f7 bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3279 read 0xcaccf7 thru 0xcb9a66 (0xcd70 bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3280 writesame 0xe23b69a thru 0xe248a7c data_size 0x93(0xd3e3 bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3281 read 0xb112aad thru 0xb11e877 (0xbdcb bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3282 read 0x25baca2 thru 0x25bcb1f (0x1e7e bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3283 read 0x582ea07 thru 0x582f9e5 (0xfdf bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3284 read 0xc30c271 thru 0xc3119a0 (0x5730 bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3287 read 0x6ee38bf thru 0x6ee58cb (0x200d bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3289 punch from 0x2174f1f to 0x21803ed, (0xb4ce bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3290 writesame 0xe4b781d thru 0xe4b9943 data_size 0x9(0x2127 bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3293 punch from 0xa3f2e5d to 0xa400c04, (0xdda7 bytes) 2024-09-13T09:27:10.342 INFO:teuthology.orchestra.run.smithi063.stdout:3294 punch from 0xb10d514 to 0xb119cb5, (0xc7a1 bytes) 2024-09-13T09:27:10.351 INFO:teuthology.orchestra.run.smithi063.stdout:3295 write 0xe77df44 thru 0xe784367 (0x6424 bytes) 2024-09-13T09:27:10.360 INFO:teuthology.orchestra.run.smithi063.stdout:3296 writesame 0x1ad8bd1 thru 0x1ade1f5 data_size 0x1(0x5625 bytes) 2024-09-13T09:27:10.379 INFO:teuthology.orchestra.run.smithi063.stdout:3297 write 0x937c736 thru 0x9388e50 (0xc71b bytes) 2024-09-13T09:27:10.386 INFO:teuthology.orchestra.run.smithi063.stdout:3298 writesame 0x361ddeb thru 0x3620be1 data_size 0x7(0x2df7 bytes) 2024-09-13T09:27:10.390 INFO:teuthology.orchestra.run.smithi063.stdout:3301 write 0xc67a22e thru 0xc684634 (0xa407 bytes) 2024-09-13T09:27:10.398 INFO:teuthology.orchestra.run.smithi063.stdout:3302 write 0xc9a8290 thru 0xc9adf21 (0x5c92 bytes) 2024-09-13T09:27:10.401 INFO:teuthology.orchestra.run.smithi063.stdout:3303 write 0x9ff8835 thru 0xa006b16 (0xe2e2 bytes) 2024-09-13T09:27:10.408 INFO:teuthology.orchestra.run.smithi063.stdout:3304 punch from 0x7215189 to 0x7220201, (0xb078 bytes) 2024-09-13T09:27:10.408 INFO:teuthology.orchestra.run.smithi063.stdout:3305 write 0x3dce948 thru 0x3dd9029 (0xa6e2 bytes) 2024-09-13T09:27:10.411 INFO:teuthology.orchestra.run.smithi063.stdout:3306 read 0x71535de thru 0x7160b02 (0xd525 bytes) 2024-09-13T09:27:10.412 INFO:teuthology.orchestra.run.smithi063.stdout:3307 write 0x5b19fe8 thru 0x5b266b6 (0xc6cf bytes) 2024-09-13T09:27:10.433 INFO:teuthology.orchestra.run.smithi063.stdout:3308 trunc from 0xe784368 to 0x67f1400 2024-09-13T09:27:10.440 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:10.431+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:10.459 INFO:teuthology.orchestra.run.smithi063.stdout:3310 clone 29 order 19 su 262144 sc 13 2024-09-13T09:27:10.492 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:10.492 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:10.492 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:10.502 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:10.540 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:10.535+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:10.604 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:10.605 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:10.605 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:10.614 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:10.641 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:10.635+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:10.721 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.756756772000003, 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:10.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.643537028000001, 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:10.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.643139012999999, 2024-09-13T09:27:10.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:10.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.968032137000002, 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.735 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:10.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.968017204999999, 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:10.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:10.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:10.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.967984098999999, 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:10.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:10.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.967967806000001, 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:10.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.967951691, 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:10.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.96792065, 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:10.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.967889026999998, 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:10.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:10.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.956995715000001, 2024-09-13T09:27:10.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:27:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.956975377999999, 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954940965999999, 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954923910000002, 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954907742, 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954694079999999, 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954674769, 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954658376000001, 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954642713999998, 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.954616359999999, 2024-09-13T09:27:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:10.813 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:10.815 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:27:10.815 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:10.815 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:10.807+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:10.892 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone28 intact 2024-09-13T09:27:10.915 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:10.907+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:10.977 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:10.977 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:10.977 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:10.977 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:10.988 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:11.033 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T09:27:11.033 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:11.027+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:11.090 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:11.090 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:11.090 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:11.090 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:11.090 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:11.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:11.117 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:11.115+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:11.211 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.4680723210000002, 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124566 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.133914+0000", 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0718118329999999, 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034024329999999998, 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124566 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133914+0000", 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133914+0000", 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133912+0000", 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133921+0000", 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8060000000000001e-06 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133939+0000", 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7994000000000001e-05 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133954+0000", 2024-09-13T09:27:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5316999999999999e-05 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.136578+0000", 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002623804 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.136662+0000", 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3831000000000004e-05 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.137317+0000", 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00065468700000000004 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124572 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.134049+0000", 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0716770000000002, 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043199029999999999, 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124572 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134049+0000", 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134049+0000", 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134048+0000", 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134053+0000", 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7539999999999997e-06 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134070+0000", 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7428e-05 2024-09-13T09:27:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134094+0000", 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3583000000000001e-05 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.137370+0000", 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0032760430000000002 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.137512+0000", 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014227700000000001 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138369+0000", 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085662900000000003 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124576 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0715667639999999, 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050917829999999999, 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124576 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134158+0000", 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134164+0000", 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.083e-06 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134180+0000", 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5177000000000001e-05 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134192+0000", 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.216e-05 2024-09-13T09:27:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138437+0000", 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042449430000000002 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138535+0000", 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7962999999999997e-05 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139251+0000", 2024-09-13T09:27:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071628499999999999 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124584 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0703962039999997, 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004962073, 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124584 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135327+0000", 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135341+0000", 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3454e-05 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135366+0000", 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.51e-05 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135378+0000", 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2048000000000001e-05 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139290+0000", 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039123580000000003 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139394+0000", 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010412999999999999 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140292+0000", 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00089737400000000004 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124586 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.070336341, 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062408630000000001, 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124586 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135389+0000", 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135396+0000", 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7309999999999996e-06 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135404+0000", 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5369999999999997e-06 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135411+0000", 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140323+0000", 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049127659999999998 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140390+0000", 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6631999999999995e-05 2024-09-13T09:27:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141630+0000", 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012403480000000001 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124592 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0702625130000003, 2024-09-13T09:27:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0076439699999999999, 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124592 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135469+0000", 2024-09-13T09:27:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135500+0000", 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1149000000000003e-05 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135513+0000", 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2771e-05 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142205+0000", 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066924239999999998 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142329+0000", 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012366100000000001 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143107+0000", 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077859999999999995 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124596 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0701507030000004, 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078700200000000001, 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124596 2024-09-13T09:27:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135574+0000", 2024-09-13T09:27:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135581+0000", 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3280000000000002e-06 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135597+0000", 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5996999999999999e-05 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135608+0000", 2024-09-13T09:27:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.166e-05 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143162+0000", 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0075534349999999998 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143306+0000", 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014448900000000001 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143445+0000", 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013898200000000001 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:124598 3.5 e78/77 3:a7b00037:::rbd_object_map.1118f3eff690:head v 78'225, mlcod=78'222)", 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.138811+0000", 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0669152190000002, 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029386799999999999, 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138811+0000", 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138811+0000", 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138810+0000", 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138820+0000", 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0876000000000001e-05 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138831+0000", 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0871e-05 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138839+0000", 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7980000000000006e-06 2024-09-13T09:27:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138953+0000", 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113396 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138996+0000", 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3504000000000002e-05 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141608+0000", 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00261203 2024-09-13T09:27:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141749+0000", 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014136199999999999 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:124599 3.5 e78/77 3:a7b00037:::rbd_object_map.1118f3eff690:head v 78'226, mlcod=78'222)", 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.140201+0000", 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0655247030000004, 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002946981, 2024-09-13T09:27:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140201+0000", 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140201+0000", 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140200+0000", 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140209+0000", 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2709999999999992e-06 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140224+0000", 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4891e-05 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140234+0000", 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0151e-05 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140575+0000", 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000341066 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140612+0000", 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7209999999999998e-05 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143043+0000", 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024307489999999998 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143148+0000", 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105419 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124608 3.7 3:e4e5f482:::rbd_data.1118f3eff690.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.0606727869999997, 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062754919999999997, 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124608 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145052+0000", 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145531+0000", 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00047855199999999998 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145542+0000", 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0961000000000001e-05 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145551+0000", 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:27:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145661+0000", 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109726 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145992+0000", 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033164000000000002 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.146845+0000", 2024-09-13T09:27:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085247199999999995 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.148693+0000", 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018483460000000001 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151235+0000", 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002542159 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151297+0000", 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1710000000000004e-05 2024-09-13T09:27:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151329+0000", 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1730000000000003e-05 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124696 3.2 3:4375ad2a:::rbd_data.1118f3eff690.00000000000000fc:head [set-alloc-hint object_size 524288 write_size 524288,write 278188~27749] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.289958+0000", 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.9157675879999996, 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029142539999999998, 2024-09-13T09:27:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124696 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.289958+0000", 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.289958+0000", 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.289956+0000", 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290010+0000", 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4321999999999999e-05 2024-09-13T09:27:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290038+0000", 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7783000000000001e-05 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290050+0000", 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2197e-05 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290170+0000", 2024-09-13T09:27:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000119833 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290535+0000", 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036538100000000002 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290932+0000", 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039636000000000001 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292509+0000", 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001576994 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292706+0000", 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000197431 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292775+0000", 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9302999999999997e-05 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292873+0000", 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7156000000000003e-05 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124700 3.1e 3:78e395d8:::rbd_data.1118f3eff690.0000000000000080:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.9034295920000002, 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011658599, 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124700 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302294+0000", 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302307+0000", 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3602e-05 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302336+0000", 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8116999999999999e-05 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302348+0000", 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2792000000000001e-05 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302394+0000", 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5797000000000002e-05 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302505+0000", 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111286 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.304334+0000", 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828325 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313641+0000", 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093073449999999999 2024-09-13T09:27:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313729+0000", 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8305000000000002e-05 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313834+0000", 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010443899999999999 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313955+0000", 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121142 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:124700 3.1e e78/77 ondisk, result = 0)", 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.8975448539999995, 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057895580000000002, 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308180+0000", 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308188+0000", 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308203+0000", 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4977000000000001e-05 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308216+0000", 2024-09-13T09:27:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2855e-05 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313698+0000", 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054822960000000002 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313716+0000", 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8167000000000001e-05 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313971+0000", 2024-09-13T09:27:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025462500000000002 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124704 3.2 3:455d9067:::rbd_data.1118f3eff690.000000000000011d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.356224+0000", 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.8495020929999999, 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035874100000000001, 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124704 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356224+0000", 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356224+0000", 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356221+0000", 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356241+0000", 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9879999999999999e-05 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356280+0000", 2024-09-13T09:27:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9159999999999998e-05 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356298+0000", 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8113000000000001e-05 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356421+0000", 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012332700000000001 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356650+0000", 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000228419 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356916+0000", 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026683799999999998 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.358141+0000", 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001224979 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.359603+0000", 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001461929 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.359710+0000", 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107086 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.359811+0000", 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100896 2024-09-13T09:27:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124732 3.b 3:d5f03ebd:::rbd_data.1118f3eff690.000000000000010a:head [set-alloc-hint object_size 524288 write_size 524288,write 470192~14750] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.404361+0000", 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.8013653590000001, 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036971999999999999, 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124732 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404361+0000", 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404361+0000", 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404359+0000", 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404386+0000", 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7631e-05 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404417+0000", 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0377000000000001e-05 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404430+0000", 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3237e-05 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404578+0000", 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000148336 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404836+0000", 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000258001 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.405068+0000", 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023155199999999999 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.406834+0000", 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017665490000000001 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.407878+0000", 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001044187 2024-09-13T09:27:11.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.407970+0000", 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1413e-05 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.408058+0000", 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7999999999999998e-05 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.294 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124738 3.18 3:186f77ea:::rbd_object_map.1118f3eff690.000000000000001f:head [writefull 0~106] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.758130+0000", 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.4475955369999998, 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003816842, 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124738 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758130+0000", 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758130+0000", 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758129+0000", 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758140+0000", 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0762e-05 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758157+0000", 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7144999999999999e-05 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758171+0000", 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4695000000000001e-05 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758244+0000", 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2460999999999996e-05 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758251+0000", 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1590000000000004e-06 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.760315+0000", 2024-09-13T09:27:11.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020640120000000001 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.760462+0000", 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000146816 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.760606+0000", 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000144198 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761833+0000", 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012271999999999999 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761854+0000", 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0903e-05 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761905+0000", 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0955000000000003e-05 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761947+0000", 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2120000000000003e-05 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125238 3.18 3:1cc4019a:::rbd_data.1118bda513e1.0000000000000005:head [stat,write 16509626~3298] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.164406096, 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065355600000000002, 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125238 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:11.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041315+0000", 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999957 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041336+0000", 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1073e-05 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041373+0000", 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7039000000000002e-05 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041394+0000", 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0477000000000001e-05 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041475+0000", 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0952999999999996e-05 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041481+0000", 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045492+0000", 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040115339999999998 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045585+0000", 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2255000000000004e-05 2024-09-13T09:27:11.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045728+0000", 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014281499999999999 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047526+0000", 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017983400000000001 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047681+0000", 2024-09-13T09:27:11.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015514200000000001 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047777+0000", 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6415000000000004e-05 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047855+0000", 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7985999999999995e-05 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125411 3.18 3:1d726b1a:::rbd_data.1118bda513e1.000000000000000c:head [set-alloc-hint object_size 16777216 write_size 16777216,write 509750~50971] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.381555+0000", 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.82417105800000001, 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003659533, 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125411 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.308 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381555+0000", 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381555+0000", 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381552+0000", 2024-09-13T09:27:11.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381640+0000", 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8102000000000005e-05 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381675+0000", 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4943000000000001e-05 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381695+0000", 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9477e-05 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381812+0000", 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117766 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.382076+0000", 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026315399999999998 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.382324+0000", 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024800100000000003 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.384111+0000", 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017869769999999999 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.385072+0000", 2024-09-13T09:27:11.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00096183100000000001 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.385157+0000", 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4672000000000005e-05 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.385214+0000", 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7365999999999999e-05 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125415 3.18 3:1d726b1a:::rbd_data.1118bda513e1.000000000000000c:head [write 6980240~23698] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.397322+0000", 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.80840443699999998, 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030380149999999998, 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125415 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.314 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397322+0000", 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397322+0000", 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397319+0000", 2024-09-13T09:27:11.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397381+0000", 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1495999999999995e-05 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397412+0000", 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1593000000000001e-05 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397426+0000", 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3457e-05 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397535+0000", 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108713 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397649+0000", 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114311 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397808+0000", 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015881599999999999 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.399193+0000", 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00138568 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.400225+0000", 2024-09-13T09:27:11.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010312539999999999 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.400306+0000", 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1664999999999998e-05 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.400360+0000", 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3140000000000003e-05 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:11.320 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:11.321 INFO:teuthology.orchestra.run.smithi063.stdout:3311 write 0xa20a95 thru 0xa2a750 (0x9cbc bytes) 2024-09-13T09:27:11.321 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:11.322 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:11.315+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:11.422 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:11.415+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:11.430 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:11.430 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:11.430 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:11.430 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:11.440 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:11.523 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:11.519+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:11.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:11.590 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:11.590 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:11.590 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:11.590 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:11.590 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:11.600 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:11.623 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:11.619+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:11.691 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:11.707 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:11.707 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:11.717 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:11.719 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:11.724 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:11.719+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:11.737 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:11.757 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:11.779 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:11.802 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:11.815 INFO:teuthology.orchestra.run.smithi063.stdout:3312 write 0x7baf4fe thru 0x7baf52b (0x2e bytes) 2024-09-13T09:27:11.821 INFO:teuthology.orchestra.run.smithi063.stdout:3314 punch from 0x5306047 to 0x530af03, (0x4ebc bytes) 2024-09-13T09:27:11.821 INFO:teuthology.orchestra.run.smithi063.stdout:3315 punch from 0x2ed433e to 0x2edc521, (0x81e3 bytes) 2024-09-13T09:27:11.821 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:11.824 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:11.819+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:11.825 INFO:teuthology.orchestra.run.smithi063.stdout:3318 trunc from 0x7baf52c to 0x50234c0 2024-09-13T09:27:11.826 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:11.834 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:11.835 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:11.847 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:11.857 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:11.860 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:11.868 INFO:teuthology.orchestra.run.smithi063.stdout:3319 trunc from 0x50234c0 to 0xd993300 2024-09-13T09:27:11.876 INFO:teuthology.orchestra.run.smithi063.stdout:3322 write 0x21f800f thru 0x2206032 (0xe024 bytes) 2024-09-13T09:27:11.904 INFO:teuthology.orchestra.run.smithi063.stdout:3323 trunc from 0xd993300 to 0xfbe558 2024-09-13T09:27:11.925 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:11.919+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:11.946 INFO:teuthology.orchestra.run.smithi063.stdout:3324 punch from 0xef6a08 to 0xf0582e, (0xee26 bytes) 2024-09-13T09:27:11.946 INFO:teuthology.orchestra.run.smithi063.stdout:3326 read 0x6df642 thru 0x6ed301 (0xdcc0 bytes) 2024-09-13T09:27:11.947 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:11.960 INFO:teuthology.orchestra.run.smithi063.stdout:3328 write 0x241a3b2 thru 0x2425a50 (0xb69f bytes) 2024-09-13T09:27:11.960 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:11.960 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:11.966 INFO:teuthology.orchestra.run.smithi063.stdout:3330 punch from 0x21cc853 to 0x21d7620, (0xadcd bytes) 2024-09-13T09:27:11.972 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:11.975 INFO:teuthology.orchestra.run.smithi063.stdout:3331 trunc from 0x2425a51 to 0x7a8a8fc 2024-09-13T09:27:12.006 INFO:teuthology.orchestra.run.smithi063.stdout:3332 write 0xea5856d thru 0xea5c812 (0x42a6 bytes) 2024-09-13T09:27:12.012 INFO:teuthology.orchestra.run.smithi063.stdout:3333 write 0xa26506a thru 0xa2692e1 (0x4278 bytes) 2024-09-13T09:27:12.026 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:12.023+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:12.080 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.080 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:12.127 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:12.123+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:12.221 INFO:teuthology.orchestra.run.smithi063.stdout:3334 write 0x559c6af thru 0x559d909 (0x125b bytes) 2024-09-13T09:27:12.227 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:12.223+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:12.228 INFO:teuthology.orchestra.run.smithi063.stdout:3335 write 0xc72325 thru 0xc770cd (0x4da9 bytes) 2024-09-13T09:27:12.235 INFO:teuthology.orchestra.run.smithi063.stdout:3339 read 0x4bf6c2d thru 0x4bf7400 (0x7d4 bytes) 2024-09-13T09:27:12.235 INFO:teuthology.orchestra.run.smithi063.stdout:3342 read 0x82f2d0 thru 0x83b4a4 (0xc1d5 bytes) 2024-09-13T09:27:12.236 INFO:teuthology.orchestra.run.smithi063.stdout:3344 trunc from 0xea5c813 to 0xea4102d 2024-09-13T09:27:12.243 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.243 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.254 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:12.278 INFO:teuthology.orchestra.run.smithi063.stdout:3345 trunc from 0xea4102d to 0x2a0b4b 2024-09-13T09:27:12.327 INFO:teuthology.orchestra.run.smithi063.stdout:3346 read 0x30bf1 thru 0x3fbc2 (0xefd2 bytes) 2024-09-13T09:27:12.328 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:12.323+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:12.337 INFO:teuthology.orchestra.run.smithi063.stdout:3348 write 0x1650955 thru 0x165d5ea (0xcc96 bytes) 2024-09-13T09:27:12.365 INFO:teuthology.orchestra.run.smithi063.stdout:3349 write 0x86cb48f thru 0x86d2869 (0x73db bytes) 2024-09-13T09:27:12.373 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.373 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.373 INFO:teuthology.orchestra.run.smithi063.stdout:3350 trunc from 0x86d286a to 0x737e737 2024-09-13T09:27:12.384 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:12.392 INFO:teuthology.orchestra.run.smithi063.stdout:3351 punch from 0x237f71b to 0x238dad0, (0xe3b5 bytes) 2024-09-13T09:27:12.392 INFO:teuthology.orchestra.run.smithi063.stdout:3352 read 0x66c5943 thru 0x66c9157 (0x3815 bytes) 2024-09-13T09:27:12.392 INFO:teuthology.orchestra.run.smithi063.stdout:3353 trunc from 0x737e737 to 0x745725f 2024-09-13T09:27:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:12.401 INFO:teuthology.orchestra.run.smithi063.stdout:3359 write 0x128da1a thru 0x128ec9c (0x1283 bytes) 2024-09-13T09:27:12.423 INFO:teuthology.orchestra.run.smithi063.stdout:3360 write 0xc64de62 thru 0xc653387 (0x5526 bytes) 2024-09-13T09:27:12.429 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:12.423+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:12.430 INFO:teuthology.orchestra.run.smithi063.stdout:3361 read 0xb75973a thru 0xb75d7b1 (0x4078 bytes) 2024-09-13T09:27:12.430 INFO:teuthology.orchestra.run.smithi063.stdout:3362 read 0x2cbf7cc thru 0x2cc1ce3 (0x2518 bytes) 2024-09-13T09:27:12.430 INFO:teuthology.orchestra.run.smithi063.stdout:3364 write 0x63e588f thru 0x63ef2fa (0x9a6c bytes) 2024-09-13T09:27:12.436 INFO:teuthology.orchestra.run.smithi063.stdout:3365 write 0x1519944 thru 0x151d490 (0x3b4d bytes) 2024-09-13T09:27:12.447 INFO:teuthology.orchestra.run.smithi063.stdout:3366 trunc from 0xc653388 to 0xe1e5485 2024-09-13T09:27:12.458 INFO:teuthology.orchestra.run.smithi063.stdout:3367 read 0x46fb110 thru 0x4707819 (0xc70a bytes) 2024-09-13T09:27:12.458 INFO:teuthology.orchestra.run.smithi063.stdout:3368 read 0x4c81e2e thru 0x4c8f711 (0xd8e4 bytes) 2024-09-13T09:27:12.458 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:12.458 INFO:teuthology.orchestra.run.smithi063.stdout:3371 writesame 0xc75273e thru 0xc761081 data_size 0x2(0xe944 bytes) 2024-09-13T09:27:12.493 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.494 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.498 INFO:teuthology.orchestra.run.smithi063.stdout:3372 trunc from 0xe1e5485 to 0xa82c320 2024-09-13T09:27:12.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:12.519 INFO:teuthology.orchestra.run.smithi063.stdout:3373 punch from 0x23e82c2 to 0x23f698e, (0xe6cc bytes) 2024-09-13T09:27:12.519 INFO:teuthology.orchestra.run.smithi063.stdout:3375 read 0x64baa6d thru 0x64c321b (0x87af bytes) 2024-09-13T09:27:12.519 INFO:teuthology.orchestra.run.smithi063.stdout:3376 read 0x1dd8c01 thru 0x1de30a3 (0xa4a3 bytes) 2024-09-13T09:27:12.535 INFO:teuthology.orchestra.run.smithi063.stdout:3378 trunc from 0xa82c320 to 0xf13d9d 2024-09-13T09:27:12.549 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:12.547+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:12.571 INFO:teuthology.orchestra.run.smithi063.stdout:3381 trunc from 0xf13d9d to 0x41adc24 2024-09-13T09:27:12.582 INFO:teuthology.orchestra.run.smithi063.stdout:3382 read 0x1fb0274 thru 0x1fb6784 (0x6511 bytes) 2024-09-13T09:27:12.582 INFO:teuthology.orchestra.run.smithi063.stdout:3383 read 0x30519a thru 0x306375 (0x11dc bytes) 2024-09-13T09:27:12.605 INFO:teuthology.orchestra.run.smithi063.stdout:3384 writesame 0xb8bd701 thru 0xb8c343b data_size 0x1(0x5d3b bytes) 2024-09-13T09:27:12.606 INFO:teuthology.orchestra.run.smithi063.stdout:3385 write 0x1f126e2 thru 0x1f1bb00 (0x941f bytes) 2024-09-13T09:27:12.612 INFO:teuthology.orchestra.run.smithi063.stdout:3386 punch from 0xa8ec9dc to 0xa8fa593, (0xdbb7 bytes) 2024-09-13T09:27:12.613 INFO:teuthology.orchestra.run.smithi063.stdout:3387 trunc from 0xb8c343c to 0xb143a05 2024-09-13T09:27:12.620 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.620 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.625 INFO:teuthology.orchestra.run.smithi063.stdout:3388 read 0x8a31f71 thru 0x8a3703f (0x50cf bytes) 2024-09-13T09:27:12.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:12.631 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:12.627+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:12.644 INFO:teuthology.orchestra.run.smithi063.stdout:3389 trunc from 0xb143a05 to 0x2873472 2024-09-13T09:27:12.657 INFO:teuthology.orchestra.run.smithi063.stdout:3390 write 0x40a3a7 thru 0x40eb63 (0x47bd bytes) 2024-09-13T09:27:12.660 INFO:teuthology.orchestra.run.smithi063.stdout:3392 punch from 0x3e510 to 0x4043d, (0x1f2d bytes) 2024-09-13T09:27:12.661 INFO:teuthology.orchestra.run.smithi063.stdout:3393 trunc from 0x2873472 to 0x1fef41b 2024-09-13T09:27:12.699 INFO:teuthology.orchestra.run.smithi063.stdout:3395 write 0x8fc2dbf thru 0x8fc2fb8 (0x1fa bytes) 2024-09-13T09:27:12.708 INFO:teuthology.orchestra.run.smithi063.stdout:3396 trunc from 0x8fc2fb9 to 0xa72098b 2024-09-13T09:27:12.721 INFO:teuthology.orchestra.run.smithi063.stdout:3397 read 0x92d15d3 thru 0x92dde31 (0xc85f bytes) 2024-09-13T09:27:12.721 INFO:teuthology.orchestra.run.smithi063.stdout:3398 write 0x96a80b5 thru 0x96b1a6c (0x99b8 bytes) 2024-09-13T09:27:12.728 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:12.728 INFO:teuthology.orchestra.run.smithi063.stdout:3401 write 0x7cd38ee thru 0x7cdcd9f (0x94b2 bytes) 2024-09-13T09:27:12.730 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:12.727+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:12.736 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.736 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:12.747 INFO:teuthology.orchestra.run.smithi063.stdout:3402 write 0xab8259c thru 0xab8c39e (0x9e03 bytes) 2024-09-13T09:27:12.753 INFO:teuthology.orchestra.run.smithi063.stdout:3403 read 0x55a31fa thru 0x55b2110 (0xef17 bytes) 2024-09-13T09:27:12.754 INFO:teuthology.orchestra.run.smithi063.stdout:3404 read 0x7e41f86 thru 0x7e463c6 (0x4441 bytes) 2024-09-13T09:27:12.754 INFO:teuthology.orchestra.run.smithi063.stdout:3408 write 0x9b0c636 thru 0x9b1bdd9 (0xf7a4 bytes) 2024-09-13T09:27:12.773 INFO:teuthology.orchestra.run.smithi063.stdout:3410 write 0xae0ab24 thru 0xae17c29 (0xd106 bytes) 2024-09-13T09:27:12.780 INFO:teuthology.orchestra.run.smithi063.stdout:3411 write 0xf2d2aa thru 0xf32a6d (0x57c4 bytes) 2024-09-13T09:27:12.782 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:12.782 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:12.782 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:27:12.782 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:27:12.786 INFO:teuthology.orchestra.run.smithi063.stdout:3412 punch from 0x3f3b712 to 0x3f484c1, (0xcdaf bytes) 2024-09-13T09:27:12.786 INFO:teuthology.orchestra.run.smithi063.stdout:3413 writesame 0x1fdecdc thru 0x1fecba1 data_size 0xf(0xdec6 bytes) 2024-09-13T09:27:12.801 INFO:teuthology.orchestra.run.smithi063.stdout:3415 write 0x88a18e thru 0x88c80e (0x2681 bytes) 2024-09-13T09:27:12.809 INFO:teuthology.orchestra.run.smithi063.stdout:3417 write 0x75dc703 thru 0x75e8d19 (0xc617 bytes) 2024-09-13T09:27:12.815 INFO:teuthology.orchestra.run.smithi063.stdout:3418 write 0xa92346 thru 0xa949b6 (0x2671 bytes) 2024-09-13T09:27:12.816 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:27:12.821 INFO:teuthology.orchestra.run.smithi063.stdout:3419 writesame 0x2eddf56 thru 0x2ede093 data_size 0x1(0x13e bytes) 2024-09-13T09:27:12.827 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:12.832 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:12.823+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:12.846 INFO:teuthology.orchestra.run.smithi063.stdout:3423 write 0xed96582 thru 0xed9f5b1 (0x9030 bytes) 2024-09-13T09:27:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:3424 read 0x4d2b761 thru 0x4d322c5 (0x6b65 bytes) 2024-09-13T09:27:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:3425 read 0x8c14182 thru 0x8c1d9b8 (0x9837 bytes) 2024-09-13T09:27:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:3426 read 0x5bdb08d thru 0x5bdd052 (0x1fc6 bytes) 2024-09-13T09:27:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:3428 read 0x582b4d7 thru 0x5837cc7 (0xc7f1 bytes) 2024-09-13T09:27:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:3430 read 0x28ea138 thru 0x28ed558 (0x3421 bytes) 2024-09-13T09:27:12.852 INFO:teuthology.orchestra.run.smithi063.stdout:3431 punch from 0x1831f44 to 0x183e2a3, (0xc35f bytes) 2024-09-13T09:27:12.871 INFO:teuthology.orchestra.run.smithi063.stdout:3432 trunc from 0xed9f5b2 to 0x3dd78d3 2024-09-13T09:27:12.899 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:12.899 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:12.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:12.910 INFO:teuthology.orchestra.run.smithi063.stdout:3433 write 0xa8f5bc4 thru 0xa8fd0e7 (0x7524 bytes) 2024-09-13T09:27:12.915 INFO:teuthology.orchestra.run.smithi063.stdout:3434 punch from 0x4af2595 to 0x4b01fa0, (0xfa0b bytes) 2024-09-13T09:27:12.915 INFO:teuthology.orchestra.run.smithi063.stdout:3436 punch from 0x719e85c to 0x71a7041, (0x87e5 bytes) 2024-09-13T09:27:12.918 INFO:teuthology.orchestra.run.smithi063.stdout:3437 trunc from 0xa8fd0e8 to 0x90a4e28 2024-09-13T09:27:12.931 INFO:teuthology.orchestra.run.smithi063.stdout:3442 read 0x74cd8b1 thru 0x74cdcba (0x40a bytes) 2024-09-13T09:27:12.932 INFO:teuthology.orchestra.run.smithi063.stdout:3445 read 0x4beb660 thru 0x4bf36c8 (0x8069 bytes) 2024-09-13T09:27:12.932 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:12.932 INFO:teuthology.orchestra.run.smithi063.stdout:3447 write 0x83d283 thru 0x8475a4 (0xa322 bytes) 2024-09-13T09:27:12.933 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:12.931+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:12.954 INFO:teuthology.orchestra.run.smithi063.stdout:3449 write 0xd206cca thru 0xd20f511 (0x8848 bytes) 2024-09-13T09:27:12.959 INFO:teuthology.orchestra.run.smithi063.stdout:3451 write 0x2e7af49 thru 0x2e84845 (0x98fd bytes) 2024-09-13T09:27:12.966 INFO:teuthology.orchestra.run.smithi063.stdout:3452 read 0x74a523e thru 0x74a8997 (0x375a bytes) 2024-09-13T09:27:12.966 INFO:teuthology.orchestra.run.smithi063.stdout:3453 write 0x2ad0622 thru 0x2ad99a4 (0x9383 bytes) 2024-09-13T09:27:12.971 INFO:teuthology.orchestra.run.smithi063.stdout:3454 write 0x625018 thru 0x627f50 (0x2f39 bytes) 2024-09-13T09:27:12.975 INFO:teuthology.orchestra.run.smithi063.stdout:3455 read 0xcac35fa thru 0xcacf6e1 (0xc0e8 bytes) 2024-09-13T09:27:12.975 INFO:teuthology.orchestra.run.smithi063.stdout:3456 writesame 0x7ec7338 thru 0x7ec77c8 data_size 0x1(0x491 bytes) 2024-09-13T09:27:12.981 INFO:teuthology.orchestra.run.smithi063.stdout:3457 write 0x6db2939 thru 0x6dbe00b (0xb6d3 bytes) 2024-09-13T09:27:12.996 INFO:teuthology.orchestra.run.smithi063.stdout:3458 write 0xdbb3882 thru 0xdbbbe86 (0x8605 bytes) 2024-09-13T09:27:13.002 INFO:teuthology.orchestra.run.smithi063.stdout:3459 punch from 0x47c7ea5 to 0x47cd394, (0x54ef bytes) 2024-09-13T09:27:13.002 INFO:teuthology.orchestra.run.smithi063.stdout:3460 write 0x5ef27cf thru 0x5ef7352 (0x4b84 bytes) 2024-09-13T09:27:13.007 INFO:teuthology.orchestra.run.smithi063.stdout:3465 read 0xa30cf46 thru 0xa31ce68 (0xff23 bytes) 2024-09-13T09:27:13.018 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.018 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.025 INFO:teuthology.orchestra.run.smithi063.stdout:3466 trunc from 0xdbbbe87 to 0x4936758 2024-09-13T09:27:13.028 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:13.042 INFO:teuthology.orchestra.run.smithi063.stdout:3467 punch from 0xd89b08 to 0xd8dbb9, (0x40b1 bytes) 2024-09-13T09:27:13.052 INFO:teuthology.orchestra.run.smithi063.stdout:3468 trunc from 0x4936758 to 0xafac875 2024-09-13T09:27:13.060 INFO:teuthology.orchestra.run.smithi063.stdout:3469 clone 30 order 24 su 2097152 sc 3 2024-09-13T09:27:13.070 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:13.067+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 84, 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "fsid": "eb7f03c3-9691-4766-8999-6f4f526cda71", 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "created": "2024-09-13T09:25:30.379173+0000", 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "modified": "2024-09-13T09:27:11.778755+0000", 2024-09-13T09:27:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "last_up_change": "2024-09-13T09:27:01.686116+0000", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "last_in_change": "2024-09-13T09:26:38.515771+0000", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_num": 5804032, 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_set": [ 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "nodeep-scrub", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "pglog_hardlimit", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "purged_snapdirs", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_deletes", 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "sortbitwise" 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_version": 6, 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:27:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_snapshot": "", 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_max": 3, 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd": 8, 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "min_compat_client": "luminous", 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "require_osd_release": "reef", 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_crimson": false, 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "pools": [ 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 1, 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "rbd", 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:41.357433+0000", 2024-09-13T09:27:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 8, 2024-09-13T09:27:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 8, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 8, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 8, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 8, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 3, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 19, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 2, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 2, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 1, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 2, 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": ".mgr", 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:42.578616+0000", 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool", 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 1, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 1, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 1, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 1, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 1, 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 0, 2024-09-13T09:27:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 0, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "options": { 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_max": 32, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_min": 1 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": {} 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 8, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 8, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 0.25, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 3, 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "pool_client.0", 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:51.228779+0000", 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 32, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 32, 2024-09-13T09:27:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 32, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 32, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 32, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "83", 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 32, 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 83, 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 1.5, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 1.5, 2024-09-13T09:27:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 1, 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 1.5, 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 1.5, 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "osds": [ 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "54ea2bd8-55b0-4ded-8a16-041261c5ba5a", 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 82, 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6816", 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6817", 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6818", 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6819", 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6822", 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6823", 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6820", 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6821", 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6817/2906107514", 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6819/2906107514", 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6823/2906107514", 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6821/2906107514", 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "0c95aee9-f871-4f6f-bbbd-1206db5d331c", 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6824", 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6825", 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6826", 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6827", 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6830", 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6831", 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6828", 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6829", 2024-09-13T09:27:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6825/3622854518", 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6827/3622854518", 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6831/3622854518", 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6829/3622854518", 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "f8253db2-d4da-46b5-95cf-cd5c9ebd7328", 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 14, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 0, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6800", 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6801", 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6802", 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6803", 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6806", 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6807", 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6804", 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6805", 2024-09-13T09:27:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6801/2105659441", 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6803/2105659441", 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6807/2105659441", 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6805/2105659441", 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "67c11a52-baeb-4ea8-89ea-26018ffdc35a", 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 15, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 46, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 74, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 48, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6808", 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6809", 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6810", 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6811", 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6814", 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6815", 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6812", 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6813", 2024-09-13T09:27:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6809/4293004354", 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6811/4293004354", 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6815/4293004354", 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6813/4293004354", 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "1cec9702-fa09-4d7c-b111-7d10b1950cfe", 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 60, 2024-09-13T09:27:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6800", 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6801", 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6802", 2024-09-13T09:27:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6803", 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6806", 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6807", 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6804", 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6805", 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6801/1789187713", 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6803/1789187713", 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1789187713", 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1789187713", 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "7c0b908a-65c9-426c-867c-cc8ed65084b3", 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 83, 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6809", 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6811", 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6813", 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6815", 2024-09-13T09:27:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6821", 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6823", 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6817", 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6819", 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6811/3410135021", 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6815/3410135021", 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3410135021", 2024-09-13T09:27:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6819/3410135021", 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "78d5990a-4535-4110-ba04-88f01ab9fe28", 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 76, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6808", 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6810", 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6812", 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6814", 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6820", 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6822", 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6816", 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6818", 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6810/4152410871", 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6814/4152410871", 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6822/4152410871", 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6818/4152410871", 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "53f8511c-729f-4423-8eac-813c01f8f72b", 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6824", 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6825", 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6826", 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6827", 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6830", 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6831", 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6828", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6829", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6825/2923799929", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6827/2923799929", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2923799929", 2024-09-13T09:27:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2923799929", 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_xinfo": [ 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.267895+0000", 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.161313+0000", 2024-09-13T09:27:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.447964+0000", 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.021320+0000", 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 48 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.883961+0000", 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.938146+0000", 2024-09-13T09:27:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.627237+0000", 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.887523+0000", 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap": [ 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "pgid": "3.12", 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "osds": [ 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: 5, 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: 6 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_items": [], 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_primaries": [], 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_temp": [], 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_temp": [], 2024-09-13T09:27:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklist": {}, 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "range_blocklist": {}, 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profiles": { 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "default": { 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-failure-domain": "osd", 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "k": "2", 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "m": "1", 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "plugin": "jerasure", 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "technique": "reed_sol_van" 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps_queue": [], 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "new_removed_snaps": [], 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "new_purged_snaps": [], 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_node_flags": {}, 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "device_class_flags": {}, 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode": { 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_enabled": false, 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_bucket_count": 0, 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_bucket": 0 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:13.209 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:13.212 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.212 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.213 INFO:tasks.thrashosds.thrasher:j is {'epoch': 84, 'fsid': 'eb7f03c3-9691-4766-8999-6f4f526cda71', 'created': '2024-09-13T09:25:30.379173+0000', 'modified': '2024-09-13T09:27:11.778755+0000', 'last_up_change': '2024-09-13T09:27:01.686116+0000', 'last_in_change': '2024-09-13T09:26:38.515771+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:25:41.357433+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:25:42.578616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:25:51.228779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '54ea2bd8-55b0-4ded-8a16-041261c5ba5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6816', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6817', 'nonce': 2906107514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6818', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6819', 'nonce': 2906107514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6822', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6823', 'nonce': 2906107514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6820', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6821', 'nonce': 2906107514}]}, 'public_addr': '172.21.15.63:6817/2906107514', 'cluster_addr': '172.21.15.63:6819/2906107514', 'heartbeat_back_addr': '172.21.15.63:6823/2906107514', 'heartbeat_front_addr': '172.21.15.63:6821/2906107514', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0c95aee9-f871-4f6f-bbbd-1206db5d331c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6824', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6825', 'nonce': 3622854518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6826', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6827', 'nonce': 3622854518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6830', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6831', 'nonce': 3622854518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6828', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6829', 'nonce': 3622854518}]}, 'public_addr': '172.21.15.63:6825/3622854518', 'cluster_addr': '172.21.15.63:6827/3622854518', 'heartbeat_back_addr': '172.21.15.63:6831/3622854518', 'heartbeat_front_addr': '172.21.15.63:6829/3622854518', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f8253db2-d4da-46b5-95cf-cd5c9ebd7328', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6800', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6801', 'nonce': 2105659441}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6802', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6803', 'nonce': 2105659441}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6806', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6807', 'nonce': 2105659441}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6804', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6805', 'nonce': 2105659441}]}, 'public_addr': '172.21.15.63:6801/2105659441', 'cluster_addr': '172.21.15.63:6803/2105659441', 'heartbeat_back_addr': '172.21.15.63:6807/2105659441', 'heartbeat_front_addr': '172.21.15.63:6805/2105659441', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '67c11a52-baeb-4ea8-89ea-26018ffdc35a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 46, 'up_from': 74, 'up_thru': 77, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6808', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6809', 'nonce': 4293004354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6810', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6811', 'nonce': 4293004354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6814', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6815', 'nonce': 4293004354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6812', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6813', 'nonce': 4293004354}]}, 'public_addr': '172.21.15.63:6809/4293004354', 'cluster_addr': '172.21.15.63:6811/4293004354', 'heartbeat_back_addr': '172.21.15.63:6815/4293004354', 'heartbeat_front_addr': '172.21.15.63:6813/4293004354', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '1cec9702-fa09-4d7c-b111-7d10b1950cfe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1789187713}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1789187713}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1789187713}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1789187713}]}, 'public_addr': '172.21.15.99:6801/1789187713', 'cluster_addr': '172.21.15.99:6803/1789187713', 'heartbeat_back_addr': '172.21.15.99:6807/1789187713', 'heartbeat_front_addr': '172.21.15.99:6805/1789187713', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7c0b908a-65c9-426c-867c-cc8ed65084b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6809', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3410135021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6813', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3410135021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6821', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3410135021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6817', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3410135021}]}, 'public_addr': '172.21.15.99:6811/3410135021', 'cluster_addr': '172.21.15.99:6815/3410135021', 'heartbeat_back_addr': '172.21.15.99:6823/3410135021', 'heartbeat_front_addr': '172.21.15.99:6819/3410135021', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78d5990a-4535-4110-ba04-88f01ab9fe28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6810', 'nonce': 4152410871}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6814', 'nonce': 4152410871}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6822', 'nonce': 4152410871}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6818', 'nonce': 4152410871}]}, 'public_addr': '172.21.15.99:6810/4152410871', 'cluster_addr': '172.21.15.99:6814/4152410871', 'heartbeat_back_addr': '172.21.15.99:6822/4152410871', 'heartbeat_front_addr': '172.21.15.99:6818/4152410871', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '53f8511c-729f-4423-8eac-813c01f8f72b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2923799929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2923799929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2923799929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2923799929}]}, 'public_addr': '172.21.15.99:6825/2923799929', 'cluster_addr': '172.21.15.99:6827/2923799929', 'heartbeat_back_addr': '172.21.15.99:6831/2923799929', 'heartbeat_front_addr': '172.21.15.99:6829/2923799929', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.267895+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.161313+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.447964+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.021320+0000', 'dead_epoch': 48}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.883961+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.938146+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.627237+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.887523+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.12', 'osds': [5, 6]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:27:13.213 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.12 2024-09-13T09:27:13.213 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.12 2024-09-13T09:27:13.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:13.214 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:13.207+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:13.312 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:13.303+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:13.324 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.324 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:13.413 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:13.407+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:13.436 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.437 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:13.514 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:13.507+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:13.552 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.552 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.562 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:13.615 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:13.611+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:13.671 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.671 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:13.725 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:13.723+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:13.787 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.787 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:13.816 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:13.807+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:13.826 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:27:13.830 INFO:teuthology.orchestra.run.smithi063.stderr:clear 3.12 pg_upmap mapping 2024-09-13T09:27:13.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:13.909 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:13.909 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:13.917 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:13.915+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:13.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:13.956 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone29 from 0xafac875 (overlap 0x2a0b4b) to 0x43454 2024-09-13T09:27:14.018 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:14.015+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:14.041 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:14.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:14.072 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:14.072 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:14.081 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:14.082 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:14.097 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T09:27:14.103 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:14.120 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:14.115+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:14.126 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:14.152 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:14.180 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:14.191 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:14.191 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:14.191 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:14.191 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:14.201 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:14.202 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:14.203 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:14.204 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:14.220 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:14.215+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:14.307 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:14.307 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:14.307 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:14.307 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:14.307 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:14.317 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:14.321 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:14.319+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:14.422 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:14.415+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:14.432 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.689307221, 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:14.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:14.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:14.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124566 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.133914+0000", 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2930467330000006, 2024-09-13T09:27:14.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034024329999999998, 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124566 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133914+0000", 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133914+0000", 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133912+0000", 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133921+0000", 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8060000000000001e-06 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133939+0000", 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7994000000000001e-05 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.133954+0000", 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5316999999999999e-05 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.136578+0000", 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002623804 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.136662+0000", 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3831000000000004e-05 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.137317+0000", 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00065468700000000004 2024-09-13T09:27:14.444 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124572 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 262144~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.134049+0000", 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2929119, 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043199029999999999, 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.445 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124572 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134049+0000", 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134049+0000", 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134048+0000", 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134053+0000", 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7539999999999997e-06 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134070+0000", 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7428e-05 2024-09-13T09:27:14.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134094+0000", 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3583000000000001e-05 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.137370+0000", 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0032760430000000002 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.137512+0000", 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014227700000000001 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138369+0000", 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085662900000000003 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124576 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.449 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2928016640000006, 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050917829999999999, 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124576 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134158+0000", 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134164+0000", 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.083e-06 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134180+0000", 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5177000000000001e-05 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134192+0000", 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.216e-05 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138437+0000", 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042449430000000002 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138535+0000", 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7962999999999997e-05 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139251+0000", 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071628499999999999 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124584 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2916311040000004, 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004962073, 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.454 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124584 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:14.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135327+0000", 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135341+0000", 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3454e-05 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135366+0000", 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.51e-05 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135378+0000", 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2048000000000001e-05 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139290+0000", 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039123580000000003 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139394+0000", 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010412999999999999 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140292+0000", 2024-09-13T09:27:14.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00089737400000000004 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124586 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2915712409999998, 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062408630000000001, 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.459 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124586 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:14.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135389+0000", 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135396+0000", 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7309999999999996e-06 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135404+0000", 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5369999999999997e-06 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135411+0000", 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140323+0000", 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049127659999999998 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140390+0000", 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6631999999999995e-05 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141630+0000", 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012403480000000001 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124592 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2914974130000001, 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0076439699999999999, 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124592 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135469+0000", 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135500+0000", 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1149000000000003e-05 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135513+0000", 2024-09-13T09:27:14.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2771e-05 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142205+0000", 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066924239999999998 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142329+0000", 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012366100000000001 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143107+0000", 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077859999999999995 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124596 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2913856030000002, 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078700200000000001, 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124596 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135574+0000", 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135581+0000", 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3280000000000002e-06 2024-09-13T09:27:14.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135597+0000", 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5996999999999999e-05 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135608+0000", 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.166e-05 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143162+0000", 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0075534349999999998 2024-09-13T09:27:14.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143306+0000", 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014448900000000001 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143445+0000", 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013898200000000001 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.472 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:124598 3.5 e78/77 3:a7b00037:::rbd_object_map.1118f3eff690:head v 78'225, mlcod=78'222)", 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.138811+0000", 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2881501190000009, 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029386799999999999, 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138811+0000", 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138811+0000", 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138810+0000", 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138820+0000", 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0876000000000001e-05 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138831+0000", 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0871e-05 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138839+0000", 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7980000000000006e-06 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138953+0000", 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113396 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138996+0000", 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3504000000000002e-05 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141608+0000", 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00261203 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141749+0000", 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014136199999999999 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:124599 3.5 e78/77 3:a7b00037:::rbd_object_map.1118f3eff690:head v 78'226, mlcod=78'222)", 2024-09-13T09:27:14.479 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.140201+0000", 2024-09-13T09:27:14.479 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2867596030000001, 2024-09-13T09:27:14.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002946981, 2024-09-13T09:27:14.479 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.479 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.480 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140201+0000", 2024-09-13T09:27:14.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140201+0000", 2024-09-13T09:27:14.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140200+0000", 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140209+0000", 2024-09-13T09:27:14.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2709999999999992e-06 2024-09-13T09:27:14.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140224+0000", 2024-09-13T09:27:14.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4891e-05 2024-09-13T09:27:14.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140234+0000", 2024-09-13T09:27:14.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0151e-05 2024-09-13T09:27:14.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140575+0000", 2024-09-13T09:27:14.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000341066 2024-09-13T09:27:14.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140612+0000", 2024-09-13T09:27:14.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7209999999999998e-05 2024-09-13T09:27:14.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143043+0000", 2024-09-13T09:27:14.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024307489999999998 2024-09-13T09:27:14.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143148+0000", 2024-09-13T09:27:14.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105419 2024-09-13T09:27:14.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.489 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.489 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.489 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124608 3.7 3:e4e5f482:::rbd_data.1118f3eff690.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.489 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:14.490 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.2819076870000004, 2024-09-13T09:27:14.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062754919999999997, 2024-09-13T09:27:14.490 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.491 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124608 2024-09-13T09:27:14.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.492 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:14.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145052+0000", 2024-09-13T09:27:14.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145531+0000", 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00047855199999999998 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145542+0000", 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0961000000000001e-05 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145551+0000", 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145661+0000", 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109726 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145992+0000", 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033164000000000002 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.146845+0000", 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085247199999999995 2024-09-13T09:27:14.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.148693+0000", 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018483460000000001 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151235+0000", 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002542159 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151297+0000", 2024-09-13T09:27:14.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1710000000000004e-05 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151329+0000", 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1730000000000003e-05 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124696 3.2 3:4375ad2a:::rbd_data.1118f3eff690.00000000000000fc:head [set-alloc-hint object_size 524288 write_size 524288,write 278188~27749] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.289958+0000", 2024-09-13T09:27:14.499 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.1370024880000003, 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029142539999999998, 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124696 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.289958+0000", 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.289958+0000", 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.289956+0000", 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290010+0000", 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4321999999999999e-05 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290038+0000", 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7783000000000001e-05 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290050+0000", 2024-09-13T09:27:14.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2197e-05 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290170+0000", 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000119833 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290535+0000", 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036538100000000002 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.290932+0000", 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039636000000000001 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292509+0000", 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001576994 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292706+0000", 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000197431 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292775+0000", 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9302999999999997e-05 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.292873+0000", 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7156000000000003e-05 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124700 3.1e 3:78e395d8:::rbd_data.1118f3eff690.0000000000000080:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.1246644920000008, 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011658599, 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124700 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302294+0000", 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302307+0000", 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3602e-05 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302336+0000", 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8116999999999999e-05 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302348+0000", 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2792000000000001e-05 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302394+0000", 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5797000000000002e-05 2024-09-13T09:27:14.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302505+0000", 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111286 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.304334+0000", 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828325 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313641+0000", 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093073449999999999 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313729+0000", 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8305000000000002e-05 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313834+0000", 2024-09-13T09:27:14.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010443899999999999 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313955+0000", 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121142 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.512 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:124700 3.1e e78/77 ondisk, result = 0)", 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.1187797540000002, 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057895580000000002, 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308180+0000", 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308188+0000", 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308203+0000", 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4977000000000001e-05 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308216+0000", 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2855e-05 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313698+0000", 2024-09-13T09:27:14.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054822960000000002 2024-09-13T09:27:14.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313716+0000", 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8167000000000001e-05 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313971+0000", 2024-09-13T09:27:14.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025462500000000002 2024-09-13T09:27:14.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.518 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.518 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124704 3.2 3:455d9067:::rbd_data.1118f3eff690.000000000000011d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.356224+0000", 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.0707369930000006, 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035874100000000001, 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124704 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.519 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356224+0000", 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356224+0000", 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356221+0000", 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356241+0000", 2024-09-13T09:27:14.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9879999999999999e-05 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356280+0000", 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9159999999999998e-05 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356298+0000", 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8113000000000001e-05 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356421+0000", 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012332700000000001 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356650+0000", 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000228419 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.356916+0000", 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026683799999999998 2024-09-13T09:27:14.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.358141+0000", 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001224979 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.359603+0000", 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001461929 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.359710+0000", 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107086 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.359811+0000", 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100896 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.525 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124732 3.b 3:d5f03ebd:::rbd_data.1118f3eff690.000000000000010a:head [set-alloc-hint object_size 524288 write_size 524288,write 470192~14750] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.404361+0000", 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.0226002590000007, 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036971999999999999, 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124732 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404361+0000", 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404361+0000", 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404359+0000", 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404386+0000", 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7631e-05 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404417+0000", 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0377000000000001e-05 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404430+0000", 2024-09-13T09:27:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3237e-05 2024-09-13T09:27:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404578+0000", 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000148336 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.404836+0000", 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000258001 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.405068+0000", 2024-09-13T09:27:14.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023155199999999999 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.406834+0000", 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017665490000000001 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.407878+0000", 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001044187 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.407970+0000", 2024-09-13T09:27:14.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1413e-05 2024-09-13T09:27:14.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.408058+0000", 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7999999999999998e-05 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124738 3.18 3:186f77ea:::rbd_object_map.1118f3eff690.000000000000001f:head [writefull 0~106] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.758130+0000", 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.6688304370000004, 2024-09-13T09:27:14.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003816842, 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124738 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758130+0000", 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758130+0000", 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758129+0000", 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:14.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758140+0000", 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0762e-05 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758157+0000", 2024-09-13T09:27:14.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7144999999999999e-05 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758171+0000", 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4695000000000001e-05 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758244+0000", 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2460999999999996e-05 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.758251+0000", 2024-09-13T09:27:14.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1590000000000004e-06 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.760315+0000", 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020640120000000001 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.760462+0000", 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000146816 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.760606+0000", 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000144198 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761833+0000", 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012271999999999999 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761854+0000", 2024-09-13T09:27:14.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0903e-05 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761905+0000", 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0955000000000003e-05 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.761947+0000", 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2120000000000003e-05 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125238 3.18 3:1cc4019a:::rbd_data.1118bda513e1.0000000000000005:head [stat,write 16509626~3298] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:14.543 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.3856409960000002, 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065355600000000002, 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.544 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125238 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041315+0000", 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999957 2024-09-13T09:27:14.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041336+0000", 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1073e-05 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041373+0000", 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7039000000000002e-05 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041394+0000", 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0477000000000001e-05 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041475+0000", 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0952999999999996e-05 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041481+0000", 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045492+0000", 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040115339999999998 2024-09-13T09:27:14.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045585+0000", 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2255000000000004e-05 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045728+0000", 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014281499999999999 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047526+0000", 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017983400000000001 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047681+0000", 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015514200000000001 2024-09-13T09:27:14.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047777+0000", 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6415000000000004e-05 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047855+0000", 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7985999999999995e-05 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.552 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125411 3.18 3:1d726b1a:::rbd_data.1118bda513e1.000000000000000c:head [set-alloc-hint object_size 16777216 write_size 16777216,write 509750~50971] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.381555+0000", 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.0454059579999999, 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003659533, 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125411 2024-09-13T09:27:14.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381555+0000", 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381555+0000", 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381552+0000", 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381640+0000", 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8102000000000005e-05 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381675+0000", 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4943000000000001e-05 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381695+0000", 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9477e-05 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.381812+0000", 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117766 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.382076+0000", 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026315399999999998 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.382324+0000", 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024800100000000003 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.384111+0000", 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017869769999999999 2024-09-13T09:27:14.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.385072+0000", 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00096183100000000001 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.385157+0000", 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4672000000000005e-05 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.385214+0000", 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7365999999999999e-05 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125415 3.18 3:1d726b1a:::rbd_data.1118bda513e1.000000000000000c:head [write 6980240~23698] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.397322+0000", 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.0296393369999999, 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030380149999999998, 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125415 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397322+0000", 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397322+0000", 2024-09-13T09:27:14.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397319+0000", 2024-09-13T09:27:14.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:14.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397381+0000", 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1495999999999995e-05 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397412+0000", 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1593000000000001e-05 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397426+0000", 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3457e-05 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397535+0000", 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108713 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397649+0000", 2024-09-13T09:27:14.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114311 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.397808+0000", 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015881599999999999 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.399193+0000", 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00138568 2024-09-13T09:27:14.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.400225+0000", 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010312539999999999 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.400306+0000", 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1664999999999998e-05 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.400360+0000", 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3140000000000003e-05 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.568 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:14.570 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:14.563+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:14.570 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:14.671 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:14.667+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.53548747699999999, 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.535530913, 2024-09-13T09:27:14.687 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for peered", 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:14.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:14.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:27:14.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 1 2024-09-13T09:27:14.692 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:14.698 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:14.773 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:14.767+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:14.806 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:14.806 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:14.806 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:14.806 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:14.806 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:14.816 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:14.873 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:14.867+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:14.969 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:14.969 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.402008393000003, 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.970 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:14.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:72716 3.12 3:4f8411ec:::rbd_data.11181839c212.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.159767344999999, 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0081225499999999992, 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.975 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 72716 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804334+0000", 2024-09-13T09:27:14.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804331+0000", 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804537+0000", 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020520799999999999 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804563+0000", 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6067999999999998e-05 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804575+0000", 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2618e-05 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804679+0000", 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104003 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.804949+0000", 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026972500000000001 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.805349+0000", 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039944600000000002 2024-09-13T09:27:14.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.808978+0000", 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036290369999999999 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812305+0000", 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033272900000000001 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812391+0000", 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6250999999999998e-05 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.812456+0000", 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5324e-05 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.981 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87135 3.1b 3:d87e56f6:::rbd_data.11181839c212.0000000000000020:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.174123344000002, 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085077469999999995, 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87135 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789975+0000", 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789989+0000", 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790012+0000", 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2889999999999999e-05 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790034+0000", 2024-09-13T09:27:14.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2221e-05 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790129+0000", 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5093000000000006e-05 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790376+0000", 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024744900000000001 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790600+0000", 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223654 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.791728+0000", 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011281170000000001 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798262+0000", 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065342899999999999 2024-09-13T09:27:14.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798385+0000", 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012293199999999999 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798486+0000", 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100324 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87861 3.1b 3:d9678466:::rbd_data.11181839c212.000000000000019b:head [set-alloc-hint object_size 262144 write_size 262144,write 179200~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.529995706000001, 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074187890000000003, 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87861 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434106+0000", 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434103+0000", 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:14.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434118+0000", 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4357e-05 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434154+0000", 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5816999999999999e-05 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434173+0000", 2024-09-13T09:27:14.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9497e-05 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434266+0000", 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2987999999999998e-05 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434491+0000", 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000224441 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:14.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.434675+0000", 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018440899999999999 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.436722+0000", 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020469669999999998 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441372+0000", 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046502059999999996 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441447+0000", 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4356000000000004e-05 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.441524+0000", 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7818000000000006e-05 2024-09-13T09:27:14.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87976 3.1b 3:dfd74b2f:::rbd_data.11181839c212.000000000000027a:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.431396105000001, 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074366880000000003, 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:14.994 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87976 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532705+0000", 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532703+0000", 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532716+0000", 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2469999999999999e-05 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532741+0000", 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5565999999999999e-05 2024-09-13T09:27:14.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532756+0000", 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4367e-05 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532900+0000", 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014460699999999999 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533180+0000", 2024-09-13T09:27:14.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027956200000000002 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533377+0000", 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019749399999999999 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535597+0000", 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022201439999999999 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.539992+0000", 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043942759999999999 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540079+0000", 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6942999999999997e-05 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540142+0000", 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3192000000000001e-05 2024-09-13T09:27:14.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87977 3.1 3:83505842:::rbd_data.11181839c212.000000000000027b:head [set-alloc-hint object_size 262144 write_size 262144,write 238592~2048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.431294122000001, 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074050050000000001, 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87977 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532807+0000", 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532806+0000", 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532815+0000", 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.346e-06 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532844+0000", 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8484e-05 2024-09-13T09:27:15.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532853+0000", 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9700000000000005e-06 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.532953+0000", 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100234 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533161+0000", 2024-09-13T09:27:15.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020798600000000001 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.533328+0000", 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016667299999999999 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.535098+0000", 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017702589999999999 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540105+0000", 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050068170000000002 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540178+0000", 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3165999999999997e-05 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.540212+0000", 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4175999999999999e-05 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.005 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.802784299999999, 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:15.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:15.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:15.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.694439111000001, 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:15.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:15.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.156094156999998, 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:15.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.155935402000001, 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.024 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:15.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:15.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.155859461999999, 2024-09-13T09:27:15.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:15.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:15.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:15.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:15.036 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.154880019, 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:15.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:15.040 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.147301298999999, 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.041 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:15.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:15.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:15.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:15.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.047 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:103778 3.1 3:868d9324:::rbd_data.11181839c212.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.353239479999999, 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080962489999999998, 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 103778 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610862+0000", 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.610860+0000", 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611180+0000", 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00032002199999999999 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611217+0000", 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6851999999999999e-05 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611235+0000", 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8003999999999999e-05 2024-09-13T09:27:15.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611307+0000", 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1868999999999999e-05 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611544+0000", 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000237042 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.611833+0000", 2024-09-13T09:27:15.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028872000000000001 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.614498+0000", 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002665813 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618843+0000", 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043449370000000001 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618891+0000", 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7885e-05 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:52.618958+0000", 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6874999999999999e-05 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:15.053 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.880555335, 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:15.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:15.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.059 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:122815 3.b 3:d285db9c:::rbd_data.1118172bd520.0000000000000013:head [stat,delete] snapc 1c=[1c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.167871975000001, 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007682538, 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 122815 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796229+0000", 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796233+0000", 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796242+0000", 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8009999999999993e-06 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.796249+0000", 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4800000000000004e-06 2024-09-13T09:27:15.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801362+0000", 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051132110000000003 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801464+0000", 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101888 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:15.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.801693+0000", 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022865299999999999 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803158+0000", 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001465343 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803777+0000", 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00061901299999999996 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803829+0000", 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1662000000000001e-05 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:59.803912+0000", 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2934000000000003e-05 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123058 3.1f 3:fceb619c:::rbd_data.1118172bd520.0000000000000007:1c [sparse-read 0~208896] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:15.065 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.559910039, 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0074653569999999997, 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123058 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:15.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404191+0000", 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404190+0000", 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404197+0000", 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0299999999999996e-06 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404214+0000", 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6612999999999998e-05 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404227+0000", 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.279e-05 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.410925+0000", 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066988780000000001 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.410985+0000", 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9876000000000001e-05 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411657+0000", 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00067126600000000001 2024-09-13T09:27:15.069 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:123059 3.1f 3:fceb619c:::rbd_data.1118172bd520.0000000000000007:1c [sparse-read 417792~106496] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.559861925, 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079657760000000008, 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.070 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 123059 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404239+0000", 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404238+0000", 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404246+0000", 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1899999999999998e-06 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404263+0000", 2024-09-13T09:27:15.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7136999999999999e-05 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.404275+0000", 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2697000000000001e-05 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411690+0000", 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.007414464 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.411751+0000", 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1230999999999997e-05 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:00.412205+0000", 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00045391099999999998 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:15.074 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.9324843039999999, 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114301, 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031615+0000", 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031627+0000", 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2072999999999999e-05 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031658+0000", 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1037000000000001e-05 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031679+0000", 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0557e-05 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040706+0000", 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090265759999999997 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040717+0000", 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1357e-05 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040731+0000", 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4355000000000001e-05 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.9323374319999997, 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114278, 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031763+0000", 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031769+0000", 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4659999999999996e-06 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031799+0000", 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9164000000000001e-05 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031812+0000", 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3465999999999999e-05 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040746+0000", 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0089339659999999998 2024-09-13T09:27:15.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040752+0000", 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7830000000000004e-06 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040878+0000", 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012623300000000001 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.083 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.083 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:15.084 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:15.085 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:15.075+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:15.185 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:15.183+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:15.242 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:15.242 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:15.242 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:15.242 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:15.252 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:15.286 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:15.283+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:15.362 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:15.362 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:15.362 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:15.362 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:15.363 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:15.372 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:15.387 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:15.383+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:15.485 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.526202556000001, 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.833789916000001, 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:27:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:27:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.833669354000001, 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.360896171, 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:15.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:15.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.512 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930543175, 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.513 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:15.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:15.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:15.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930517300999998, 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:15.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.522 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930491936999999, 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:15.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.527 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930406914999999, 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:15.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.533 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.93037352, 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:15.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:15.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930359582000001, 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:15.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:15.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930321836000001, 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.93030392, 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.549 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930290272000001, 2024-09-13T09:27:15.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:15.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930223627, 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930175566999999, 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.568 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930118295, 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:15.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930098185999999, 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.577 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930076039999999, 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:15.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930019773000001, 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.587 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.930003877000001, 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:15.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.593 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:15.594 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:15.595 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:15.591+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:15.696 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:15.691+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:15.709 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:15.710 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:15.710 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:15.710 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:15.720 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:15.796 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:15.791+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:15.831 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:15.831 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:15.831 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:15.831 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:15.831 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:15.840 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:27:15.843 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:15.897 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:15.891+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:15.948 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.139373544999998, 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:27:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 72.731197660000007, 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:27:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 72.731007536999996, 2024-09-13T09:27:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:27:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 72.730195128000005, 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.541288479000002, 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:27:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:27:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.976179094000003, 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.902640081000001, 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.810096712999993, 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:27:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.782036136999999, 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.767778354000001, 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.299059901, 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:27:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:27:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.310135207999998, 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.105109159999998, 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.006771112000003, 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:27:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:27:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.005962507, 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:27:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.931369936999999, 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.737287494, 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.621665995000001, 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:27:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:27:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.352924652999999, 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:16.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:106878 3.4 e69/21 3:2395b47f:::rbd_data.1118fd5aef74.0000000000000000:head v 69'245, mlcod=68'243)", 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.962949538, 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002931983, 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979362+0000", 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980127+0000", 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00076519900000000004 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980141+0000", 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4419999999999999e-05 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980170+0000", 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8915e-05 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980224+0000", 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3940000000000002e-05 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980243+0000", 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9210000000000001e-05 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982244+0000", 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020012350000000001 2024-09-13T09:27:16.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982295+0000", 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0101e-05 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:16.039 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:16.041 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:16.041 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:16.031+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:16.141 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:16.139+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:16.163 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:16.164 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:16.164 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:16.164 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:16.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:16.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:16.219 INFO:teuthology.orchestra.run.smithi063.stdout:3472 read 0x3e1855f thru 0x3e1867d (0x11f bytes) 2024-09-13T09:27:16.227 INFO:teuthology.orchestra.run.smithi063.stdout:3473 read 0x7411799 thru 0x7418c1f (0x7487 bytes) 2024-09-13T09:27:16.228 INFO:teuthology.orchestra.run.smithi063.stdout:3474 punch from 0x464c3f1 to 0x4656047, (0x9c56 bytes) 2024-09-13T09:27:16.228 INFO:teuthology.orchestra.run.smithi063.stdout:3475 punch from 0x61ceb5d to 0x61d5da1, (0x7244 bytes) 2024-09-13T09:27:16.228 INFO:teuthology.orchestra.run.smithi063.stdout:3477 read 0x1f4972f thru 0x1f4b87e (0x2150 bytes) 2024-09-13T09:27:16.244 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:16.239+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:16.255 INFO:teuthology.orchestra.run.smithi063.stdout:3478 writesame 0xede7c2e thru 0xedf76cb data_size 0x6f(0xfa9e bytes) 2024-09-13T09:27:16.263 INFO:teuthology.orchestra.run.smithi063.stdout:3479 write 0xee19054 thru 0xee1b51b (0x24c8 bytes) 2024-09-13T09:27:16.268 INFO:teuthology.orchestra.run.smithi063.stdout:3480 write 0x934ce55 thru 0x93547df (0x798b bytes) 2024-09-13T09:27:16.274 INFO:teuthology.orchestra.run.smithi063.stdout:3481 write 0x5469090 thru 0x54740c7 (0xb038 bytes) 2024-09-13T09:27:16.281 INFO:teuthology.orchestra.run.smithi063.stdout:3483 write 0x673176f thru 0x673d723 (0xbfb5 bytes) 2024-09-13T09:27:16.286 INFO:teuthology.orchestra.run.smithi063.stdout:3484 writesame 0x448e130 thru 0x44986fa data_size 0x1(0xa5cb bytes) 2024-09-13T09:27:16.300 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:16.300 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:16.300 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:16.300 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:16.300 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:16.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:16.316 INFO:teuthology.orchestra.run.smithi063.stdout:3485 write 0xd241a64 thru 0xd24914a (0x76e7 bytes) 2024-09-13T09:27:16.322 INFO:teuthology.orchestra.run.smithi063.stdout:3486 writesame 0xb5a0e09 thru 0xb5ad496 data_size 0x82(0xc68e bytes) 2024-09-13T09:27:16.331 INFO:teuthology.orchestra.run.smithi063.stdout:3487 trunc from 0xee1b51c to 0xd1d7b93 2024-09-13T09:27:16.342 INFO:teuthology.orchestra.run.smithi063.stdout:3490 write 0x1203d8 thru 0x123676 (0x329f bytes) 2024-09-13T09:27:16.349 INFO:teuthology.orchestra.run.smithi063.stdout:3492 trunc from 0xd1d7b93 to 0xed8474d 2024-09-13T09:27:16.357 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:16.355+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:16.369 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:16.373 INFO:teuthology.orchestra.run.smithi063.stdout:3495 trunc from 0xed8474d to 0x7b6a465 2024-09-13T09:27:16.393 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:16.397 INFO:teuthology.orchestra.run.smithi063.stdout:3496 punch from 0x2d6b19e to 0x2d6bb85, (0x9e7 bytes) 2024-09-13T09:27:16.397 INFO:teuthology.orchestra.run.smithi063.stdout:3498 read 0x6ea4076 thru 0x6ea436b (0x2f6 bytes) 2024-09-13T09:27:16.398 INFO:teuthology.orchestra.run.smithi063.stdout:3499 writesame 0xd80903 thru 0xd9083f data_size 0x13(0xff3d bytes) 2024-09-13T09:27:16.404 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.413 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:16.418 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:16.419 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:16.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:16.419 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:16.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.419 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.419 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721751556000001, 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.420 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:16.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.424 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721729042, 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:16.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:16.429 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721713205, 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:16.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:16.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721516261000001, 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:16.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:16.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721491937, 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:16.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:16.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:16.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721475476999998, 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:16.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721459456000002, 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:16.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721409123000001, 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:16.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:16.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.721305548, 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:16.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:16.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711927752000001, 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:16.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:16.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711909087999999, 2024-09-13T09:27:16.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:16.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711892597999999, 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711876813, 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:16.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711860691999998, 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:16.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:16.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711844787, 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711828317999998, 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:16.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:16.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711806465999999, 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:16.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711789651, 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:16.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:16.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711773354000002, 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:16.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:16.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:16.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.711757753000001, 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:16.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:16.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:16.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:16.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:16.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:16.520 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:16.521 INFO:teuthology.orchestra.run.smithi063.stdout:3504 writesame 0xb0dc461 thru 0xb0e992e data_size 0x1(0xd4ce bytes) 2024-09-13T09:27:16.521 INFO:teuthology.orchestra.run.smithi063.stdout:3506 punch from 0x5e48fcb to 0x5e57fcb, (0xf000 bytes) 2024-09-13T09:27:16.522 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:16.522 INFO:teuthology.orchestra.run.smithi063.stdout:3507 trunc from 0xb0e992f to 0x1d7092e 2024-09-13T09:27:16.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:16.523 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:16.523 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.524 INFO:teuthology.orchestra.run.smithi063.stdout:3510 punch from 0x11e015a to 0x11ec320, (0xc1c6 bytes) 2024-09-13T09:27:16.524 INFO:teuthology.orchestra.run.smithi063.stdout:3511 read 0x102268 thru 0x104fe8 (0x2d81 bytes) 2024-09-13T09:27:16.524 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:16.524 INFO:teuthology.orchestra.run.smithi063.stdout:3512 writesame 0xb2ef716 thru 0xb2f2fb9 data_size 0x32(0x38a4 bytes) 2024-09-13T09:27:16.525 INFO:teuthology.orchestra.run.smithi063.stdout:3513 trunc from 0xb2f2fba to 0x65d9af5 2024-09-13T09:27:16.525 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:16.525 INFO:teuthology.orchestra.run.smithi063.stdout:3514 punch from 0x35354f3 to 0x3541b44, (0xc651 bytes) 2024-09-13T09:27:16.525 INFO:teuthology.orchestra.run.smithi063.stdout:3516 read 0x52f10a0 thru 0x52ff34e (0xe2af bytes) 2024-09-13T09:27:16.526 INFO:teuthology.orchestra.run.smithi063.stdout:3519 writesame 0x74e5ad thru 0x74f903 data_size 0x1(0x1357 bytes) 2024-09-13T09:27:16.526 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:16.515+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:16.536 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:16.539 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:16.539 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:16.539 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:16.539 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:16.539 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:16.540 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:16.540 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:16.540 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:16.541 INFO:teuthology.orchestra.run.smithi063.stdout:3520 write 0xd3473a6 thru 0xd34d2b8 (0x5f13 bytes) 2024-09-13T09:27:16.547 INFO:teuthology.orchestra.run.smithi063.stdout:3521 read 0x7330173 thru 0x73359ba (0x5848 bytes) 2024-09-13T09:27:16.547 INFO:teuthology.orchestra.run.smithi063.stdout:3523 read 0x92d4fa8 thru 0x92e3a87 (0xeae0 bytes) 2024-09-13T09:27:16.548 INFO:teuthology.orchestra.run.smithi063.stdout:3524 trunc from 0xd34d2b9 to 0xc976cab 2024-09-13T09:27:16.560 INFO:teuthology.orchestra.run.smithi063.stdout:3525 writesame 0x8ded1de thru 0x8deeb44 data_size 0x1(0x1967 bytes) 2024-09-13T09:27:16.569 INFO:teuthology.orchestra.run.smithi063.stdout:3526 writesame 0x9af9365 thru 0x9b02d16 data_size 0x2(0x99b2 bytes) 2024-09-13T09:27:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:3527 read 0xb1b779a thru 0xb1bc8fb (0x5162 bytes) 2024-09-13T09:27:16.592 INFO:teuthology.orchestra.run.smithi063.stdout:3528 read 0xb0be548 thru 0xb0be835 (0x2ee bytes) 2024-09-13T09:27:16.596 INFO:teuthology.orchestra.run.smithi063.stdout:3529 trunc from 0xc976cab to 0xe537a9b 2024-09-13T09:27:16.608 INFO:teuthology.orchestra.run.smithi063.stdout:3530 writesame 0xc52516a thru 0xc5257d6 data_size 0x7(0x66d bytes) 2024-09-13T09:27:16.613 INFO:teuthology.orchestra.run.smithi063.stdout:3531 read 0x7790400 thru 0x7792a9a (0x269b bytes) 2024-09-13T09:27:16.613 INFO:teuthology.orchestra.run.smithi063.stdout:3533 write 0x20cbf18 thru 0x20cf624 (0x370d bytes) 2024-09-13T09:27:16.616 INFO:teuthology.orchestra.run.smithi063.stdout:3534 writesame 0x60c4944 thru 0x60cd116 data_size 0xb(0x87d3 bytes) 2024-09-13T09:27:16.621 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:16.619+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:16.623 INFO:teuthology.orchestra.run.smithi063.stdout:3535 write 0x32f2de9 thru 0x32fef53 (0xc16b bytes) 2024-09-13T09:27:16.630 INFO:teuthology.orchestra.run.smithi063.stdout:3536 read 0xc188a1c thru 0xc19349d (0xaa82 bytes) 2024-09-13T09:27:16.632 INFO:teuthology.orchestra.run.smithi063.stdout:3537 trunc from 0xe537a9b to 0xdc91ad1 2024-09-13T09:27:16.643 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:16.644 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:16.646 INFO:teuthology.orchestra.run.smithi063.stdout:3538 writesame 0x30d5565 thru 0x30db281 data_size 0xb(0x5d1d bytes) 2024-09-13T09:27:16.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:16.655 INFO:teuthology.orchestra.run.smithi063.stdout:3539 punch from 0x2498c0 to 0x24b9b2, (0x20f2 bytes) 2024-09-13T09:27:16.681 INFO:teuthology.orchestra.run.smithi063.stdout:3540 trunc from 0xdc91ad1 to 0x1a7cfb4 2024-09-13T09:27:16.723 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:16.715+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:16.743 INFO:teuthology.orchestra.run.smithi063.stdout:3541 writesame 0xd971a62 thru 0xd97a4fb data_size 0x71(0x8a9a bytes) 2024-09-13T09:27:16.744 INFO:teuthology.orchestra.run.smithi063.stdout:3542 write 0xb6afab3 thru 0xb6bafcc (0xb51a bytes) 2024-09-13T09:27:16.751 INFO:teuthology.orchestra.run.smithi063.stdout:3544 punch from 0x498fa7a to 0x499ea2f, (0xefb5 bytes) 2024-09-13T09:27:16.751 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.751 INFO:teuthology.orchestra.run.smithi063.stdout:3546 write 0x591c0f9 thru 0x5929965 (0xd86d bytes) 2024-09-13T09:27:16.759 INFO:teuthology.orchestra.run.smithi063.stdout:3547 write 0x4b71e37 thru 0x4b7b1c9 (0x9393 bytes) 2024-09-13T09:27:16.763 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:16.763 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:16.766 INFO:teuthology.orchestra.run.smithi063.stdout:3548 punch from 0x441bac8 to 0x4429368, (0xd8a0 bytes) 2024-09-13T09:27:16.766 INFO:teuthology.orchestra.run.smithi063.stdout:3549 write 0xa549100 thru 0xa54edeb (0x5cec bytes) 2024-09-13T09:27:16.770 INFO:teuthology.orchestra.run.smithi063.stdout:3550 punch from 0xa1ac6fe to 0xa1b57df, (0x90e1 bytes) 2024-09-13T09:27:16.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:16.785 INFO:teuthology.orchestra.run.smithi063.stdout:3552 writesame 0xed49e7c thru 0xed55753 data_size 0x46(0xb8d8 bytes) 2024-09-13T09:27:16.786 INFO:teuthology.orchestra.run.smithi063.stdout:3553 read 0x4be7ffa thru 0x4bf4abe (0xcac5 bytes) 2024-09-13T09:27:16.786 INFO:teuthology.orchestra.run.smithi063.stdout:3554 read 0xd5342b2 thru 0xd53d481 (0x91d0 bytes) 2024-09-13T09:27:16.786 INFO:teuthology.orchestra.run.smithi063.stdout:3555 write 0x54a7d0a thru 0x54b5203 (0xd4fa bytes) 2024-09-13T09:27:16.794 INFO:teuthology.orchestra.run.smithi063.stdout:3556 write 0xd00cc2a thru 0xd01685a (0x9c31 bytes) 2024-09-13T09:27:16.800 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.800 INFO:teuthology.orchestra.run.smithi063.stdout:3559 read 0x40cae2b thru 0x40d3766 (0x893c bytes) 2024-09-13T09:27:16.800 INFO:teuthology.orchestra.run.smithi063.stdout:3560 write 0x478b469 thru 0x4794804 (0x939c bytes) 2024-09-13T09:27:16.804 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.823 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:16.815+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:16.827 INFO:teuthology.orchestra.run.smithi063.stdout:3562 trunc from 0xed55754 to 0x445374c 2024-09-13T09:27:16.845 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:27:16.858 INFO:teuthology.orchestra.run.smithi063.stdout:3563 read 0x43481f7 thru 0x434f70c (0x7516 bytes) 2024-09-13T09:27:16.871 INFO:teuthology.orchestra.run.smithi063.stdout:3565 write 0x6ca5e12 thru 0x6caf25a (0x9449 bytes) 2024-09-13T09:27:16.877 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.877 INFO:teuthology.orchestra.run.smithi063.stdout:3567 read 0x5e6c72 thru 0x5f0a4e (0x9ddd bytes) 2024-09-13T09:27:16.878 INFO:teuthology.orchestra.run.smithi063.stdout:3568 writesame 0x14baec5 thru 0x14be76d data_size 0x5(0x38a9 bytes) 2024-09-13T09:27:16.883 INFO:teuthology.orchestra.run.smithi063.stdout:3569 write 0x2d8518c thru 0x2d8e6ed (0x9562 bytes) 2024-09-13T09:27:16.884 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:16.884 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:16.885 INFO:teuthology.orchestra.run.smithi063.stdout:3570 writesame 0x1c59536 thru 0x1c60875 data_size 0x20(0x7340 bytes) 2024-09-13T09:27:16.889 INFO:teuthology.orchestra.run.smithi063.stdout:3571 read 0x2ce76e8 thru 0x2ce871a (0x1033 bytes) 2024-09-13T09:27:16.890 INFO:teuthology.orchestra.run.smithi063.stdout:3572 writesame 0x2ff6843 thru 0x3003904 data_size 0x9(0xd0c2 bytes) 2024-09-13T09:27:16.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:16.897 INFO:teuthology.orchestra.run.smithi063.stdout:3573 read 0x368bf60 thru 0x369b603 (0xf6a4 bytes) 2024-09-13T09:27:16.898 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.898 INFO:teuthology.orchestra.run.smithi063.stdout:3575 read 0xc14d60 thru 0xc22558 (0xd7f9 bytes) 2024-09-13T09:27:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.900 INFO:teuthology.orchestra.run.smithi063.stdout:3578 read 0x62a037 thru 0x638557 (0xe521 bytes) 2024-09-13T09:27:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:3579 read 0x5428dfe thru 0x5438703 (0xf906 bytes) 2024-09-13T09:27:16.903 INFO:teuthology.orchestra.run.smithi063.stdout:3580 writesame 0x226c9b1 thru 0x226d7f4 data_size 0x16(0xe44 bytes) 2024-09-13T09:27:16.906 INFO:teuthology.orchestra.run.smithi063.stdout:3581 punch from 0x243a7b5 to 0x2447b83, (0xd3ce bytes) 2024-09-13T09:27:16.906 INFO:teuthology.orchestra.run.smithi063.stdout:3582 read 0x61d3118 thru 0x61dda72 (0xa95b bytes) 2024-09-13T09:27:16.907 INFO:teuthology.orchestra.run.smithi063.stdout:3583 punch from 0xa04779 to 0xa0a246, (0x5acd bytes) 2024-09-13T09:27:16.924 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:16.919+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:16.928 INFO:teuthology.orchestra.run.smithi063.stdout:3584 write 0xa8c9ce6 thru 0xa8cdb97 (0x3eb2 bytes) 2024-09-13T09:27:16.934 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:16.935 INFO:teuthology.orchestra.run.smithi063.stdout:3587 punch from 0x9de800 to 0x9dead9, (0x2d9 bytes) 2024-09-13T09:27:16.935 INFO:teuthology.orchestra.run.smithi063.stdout:3588 read 0x9b3976d thru 0x9b419fa (0x828e bytes) 2024-09-13T09:27:16.935 INFO:teuthology.orchestra.run.smithi063.stdout:3589 writesame 0xa60c6ac thru 0xa6161ca data_size 0x5d(0x9b1f bytes) 2024-09-13T09:27:16.950 INFO:teuthology.orchestra.run.smithi063.stdout:3590 trunc from 0xa8cdb98 to 0x770f2f0 2024-09-13T09:27:16.974 INFO:teuthology.orchestra.run.smithi063.stdout:3591 write 0x2eb7ae1 thru 0x2ec7a8c (0xffac bytes) 2024-09-13T09:27:16.977 INFO:teuthology.orchestra.run.smithi063.stdout:3592 read 0x2af161d thru 0x2af343f (0x1e23 bytes) 2024-09-13T09:27:16.996 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:16.996 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.004 INFO:teuthology.orchestra.run.smithi063.stdout:3593 write 0xdf13597 thru 0xdf1c877 (0x92e1 bytes) 2024-09-13T09:27:17.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:17.011 INFO:teuthology.orchestra.run.smithi063.stdout:3596 clone 31 order 23 su 4194304 sc 11 2024-09-13T09:27:17.025 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:17.019+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:17.111 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.111 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:17.126 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:17.119+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:17.227 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:17.219+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:17.268 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.268 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.278 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:17.328 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:17.323+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:17.383 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.384 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:17.438 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:17.435+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:17.500 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.500 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.510 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:17.530 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:17.527+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:17.615 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.615 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.625 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:17.631 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:17.627+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:17.732 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:17.727+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:17.743 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.743 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.754 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:17.833 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:17.831+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:17.866 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.866 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:17.934 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:17.931+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:17.991 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:17.991 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:17.995 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone30 from 0xdf1c878 (overlap 0x1a7cfb4) to 0x128aafe 2024-09-13T09:27:18.001 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:18.035 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:18.031+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:18.107 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:18.107 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:18.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:18.136 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:18.127+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:18.169 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T09:27:18.185 INFO:teuthology.orchestra.run.smithi063.stdout:3597 punch from 0x2d836f4 to 0x2d84082, (0x98e bytes) 2024-09-13T09:27:18.192 INFO:teuthology.orchestra.run.smithi063.stdout:3598 writesame 0x88c8f50 thru 0x88d3027 data_size 0x4(0xa0d8 bytes) 2024-09-13T09:27:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:3599 punch from 0xbade03f to 0xbae225b, (0x421c bytes) 2024-09-13T09:27:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:3601 punch from 0x28b3dbb to 0x28b681b, (0x2a60 bytes) 2024-09-13T09:27:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:3602 read 0x4958564 thru 0x49633b4 (0xae51 bytes) 2024-09-13T09:27:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:3603 read 0xbc1005b thru 0xbc1d1ae (0xd154 bytes) 2024-09-13T09:27:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:3604 read 0x47d791a thru 0x47e4412 (0xcaf9 bytes) 2024-09-13T09:27:18.211 INFO:teuthology.orchestra.run.smithi063.stdout:3605 trunc from 0xdf1c878 to 0xe2fc719 2024-09-13T09:27:18.228 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:18.228 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:18.237 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:18.235+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:18.242 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:18.249 INFO:teuthology.orchestra.run.smithi063.stdout:3606 trunc from 0xe2fc719 to 0x1a8dd6b 2024-09-13T09:27:18.294 INFO:teuthology.orchestra.run.smithi063.stdout:3607 trunc from 0x1a8dd6b to 0x207fc0d 2024-09-13T09:27:18.306 INFO:teuthology.orchestra.run.smithi063.stdout:3608 flatten 2024-09-13T09:27:18.317 INFO:teuthology.orchestra.run.smithi063.stdout:3609 punch from 0x19abb60 to 0x19b56a6, (0x9b46 bytes) 2024-09-13T09:27:18.337 INFO:teuthology.orchestra.run.smithi063.stdout:3610 write 0x617d318 thru 0x6184d9f (0x7a88 bytes) 2024-09-13T09:27:18.338 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:18.331+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:18.342 INFO:teuthology.orchestra.run.smithi063.stdout:3611 write 0x552b11c thru 0x55369f9 (0xb8de bytes) 2024-09-13T09:27:18.349 INFO:teuthology.orchestra.run.smithi063.stdout:3612 punch from 0x2a31f8b to 0x2a3cd18, (0xad8d bytes) 2024-09-13T09:27:18.349 INFO:teuthology.orchestra.run.smithi063.stdout:3614 read 0x5fe6398 thru 0x5ff3796 (0xd3ff bytes) 2024-09-13T09:27:18.366 INFO:teuthology.orchestra.run.smithi063.stdout:3615 write 0x97246d0 thru 0x97319e7 (0xd318 bytes) 2024-09-13T09:27:18.367 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:18.368 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:18.373 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.373 INFO:teuthology.orchestra.run.smithi063.stdout:3618 read 0x7d5d4f7 thru 0x7d67d8f (0xa899 bytes) 2024-09-13T09:27:18.374 INFO:teuthology.orchestra.run.smithi063.stdout:3620 write 0x6eb921 thru 0x6f8586 (0xcc66 bytes) 2024-09-13T09:27:18.376 INFO:teuthology.orchestra.run.smithi063.stdout:3622 read 0x3b1464e thru 0x3b1da43 (0x93f6 bytes) 2024-09-13T09:27:18.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:18.379 INFO:teuthology.orchestra.run.smithi063.stdout:3623 trunc from 0x97319e8 to 0x8fe109d 2024-09-13T09:27:18.390 INFO:teuthology.orchestra.run.smithi063.stdout:3625 read 0x4840165 thru 0x484088f (0x72b bytes) 2024-09-13T09:27:18.391 INFO:teuthology.orchestra.run.smithi063.stdout:3626 read 0x6f5b35e thru 0x6f6455e (0x9201 bytes) 2024-09-13T09:27:18.413 INFO:teuthology.orchestra.run.smithi063.stdout:3627 writesame 0xe576a72 thru 0xe57bfa0 data_size 0x3(0x552f bytes) 2024-09-13T09:27:18.413 INFO:teuthology.orchestra.run.smithi063.stdout:3628 punch from 0x7e9fc05 to 0x7ead5e1, (0xd9dc bytes) 2024-09-13T09:27:18.413 INFO:teuthology.orchestra.run.smithi063.stdout:3629 punch from 0xd73de36 to 0xd7412bd, (0x3487 bytes) 2024-09-13T09:27:18.413 INFO:teuthology.orchestra.run.smithi063.stdout:3630 writesame 0x133acb3 thru 0x13476ab data_size 0x9(0xc9f9 bytes) 2024-09-13T09:27:18.439 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:18.435+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:18.442 INFO:teuthology.orchestra.run.smithi063.stdout:3631 trunc from 0xe57bfa1 to 0x15d95fe 2024-09-13T09:27:18.493 INFO:teuthology.orchestra.run.smithi063.stdout:3633 write 0x616e0c8 thru 0x617a691 (0xc5ca bytes) 2024-09-13T09:27:18.499 INFO:teuthology.orchestra.run.smithi063.stdout:3635 read 0x5f41733 thru 0x5f46f4b (0x5819 bytes) 2024-09-13T09:27:18.514 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:18.514 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:18.516 INFO:teuthology.orchestra.run.smithi063.stdout:3637 write 0x9ada257 thru 0x9ae3803 (0x95ad bytes) 2024-09-13T09:27:18.525 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:18.526 INFO:teuthology.orchestra.run.smithi063.stdout:3638 punch from 0x7369444 to 0x736ebcc, (0x5788 bytes) 2024-09-13T09:27:18.540 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:18.531+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:3641 writesame 0xdc86ce0 thru 0xdc93745 data_size 0x7(0xca66 bytes) 2024-09-13T09:27:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:3643 punch from 0x33c3b74 to 0x33d29a7, (0xee33 bytes) 2024-09-13T09:27:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:3645 read 0x38d1089 thru 0x38d8fe5 (0x7f5d bytes) 2024-09-13T09:27:18.551 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:18.556 INFO:teuthology.orchestra.run.smithi063.stdout:3647 write 0xeb822fa thru 0xeb90f8c (0xec93 bytes) 2024-09-13T09:27:18.563 INFO:teuthology.orchestra.run.smithi063.stdout:3649 read 0x817d907 thru 0x8188ed8 (0xb5d2 bytes) 2024-09-13T09:27:18.573 INFO:teuthology.orchestra.run.smithi063.stdout:3650 trunc from 0xeb90f8d to 0x9855b80 2024-09-13T09:27:18.611 INFO:teuthology.orchestra.run.smithi063.stdout:3651 trunc from 0x9855b80 to 0xef5dc1 2024-09-13T09:27:18.641 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:18.635+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:18.647 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:18.647 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:18.661 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:18.663 INFO:teuthology.orchestra.run.smithi063.stdout:3652 write 0xa7ba29f thru 0xa7c1a2d (0x778f bytes) 2024-09-13T09:27:18.670 INFO:teuthology.orchestra.run.smithi063.stdout:3653 read 0x87ebff4 thru 0x87ed8ee (0x18fb bytes) 2024-09-13T09:27:18.670 INFO:teuthology.orchestra.run.smithi063.stdout:3654 read 0x50cbdb6 thru 0x50d371f (0x796a bytes) 2024-09-13T09:27:18.684 INFO:teuthology.orchestra.run.smithi063.stdout:3655 trunc from 0xa7c1a2e to 0x2d9ca5c 2024-09-13T09:27:18.697 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:18.706 INFO:teuthology.orchestra.run.smithi063.stdout:3656 punch from 0x1282f27 to 0x12906a9, (0xd782 bytes) 2024-09-13T09:27:18.706 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.706 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.706 INFO:teuthology.orchestra.run.smithi063.stdout:3659 read 0x743524 thru 0x744379 (0xe56 bytes) 2024-09-13T09:27:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:3662 punch from 0x20866e6 to 0x208fb29, (0x9443 bytes) 2024-09-13T09:27:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:3663 read 0x86d33d thru 0x879dc5 (0xca89 bytes) 2024-09-13T09:27:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:3664 read 0x136d841 thru 0x1379c36 (0xc3f6 bytes) 2024-09-13T09:27:18.707 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.719 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:18.727 INFO:teuthology.orchestra.run.smithi063.stdout:3666 writesame 0x9790b34 thru 0x979d212 data_size 0x7(0xc6df bytes) 2024-09-13T09:27:18.727 INFO:teuthology.orchestra.run.smithi063.stdout:3667 punch from 0x1572d09 to 0x1577310, (0x4607 bytes) 2024-09-13T09:27:18.739 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:18.741 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:18.739+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:18.744 INFO:teuthology.orchestra.run.smithi063.stdout:3668 writesame 0xd8d929a thru 0xd8e7237 data_size 0x3(0xdf9e bytes) 2024-09-13T09:27:18.744 INFO:teuthology.orchestra.run.smithi063.stdout:3669 read 0xcee3ac4 thru 0xceec039 (0x8576 bytes) 2024-09-13T09:27:18.745 INFO:teuthology.orchestra.run.smithi063.stdout:3670 write 0x13fe6e3 thru 0x14053d6 (0x6cf4 bytes) 2024-09-13T09:27:18.754 INFO:teuthology.orchestra.run.smithi063.stdout:3671 writesame 0x6f8c321 thru 0x6f903ed data_size 0x1(0x40cd bytes) 2024-09-13T09:27:18.760 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:18.773 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.773 INFO:teuthology.orchestra.run.smithi063.stdout:3674 read 0x53e9408 thru 0x53eddeb (0x49e4 bytes) 2024-09-13T09:27:18.773 INFO:teuthology.orchestra.run.smithi063.stdout:3675 writesame 0xad99f58 thru 0xada81e0 data_size 0xd(0xe289 bytes) 2024-09-13T09:27:18.780 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:18.781 INFO:teuthology.orchestra.run.smithi063.stdout:3676 read 0x894d2ff thru 0x895a3e6 (0xd0e8 bytes) 2024-09-13T09:27:18.782 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.783 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:18.783 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:18.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:18.783 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:18.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 68.826072269999997, 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:18.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:18.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:18.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:18.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.133659629999997, 2024-09-13T09:27:18.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:27:18.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:27:18.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.133539067999997, 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:18.794 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 40.660765885000004, 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:18.798 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:18.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:18.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230412889, 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.804 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:18.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230387014999998, 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:18.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230361651000003, 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.815 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:18.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230276629000002, 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.820 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:18.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:18.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.825 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230243234, 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:18.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.830 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230229295999997, 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.831 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:18.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:18.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:18.835 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230191550000001, 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:18.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:18.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:18.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230173634000003, 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.840 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:18.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.845 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.845 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:18.845 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230159985999997, 2024-09-13T09:27:18.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:18.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230093341, 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:18.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:18.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:18.855 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.230045281000002, 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:18.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:18.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:18.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:18.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:18.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.229988009000003, 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:18.863 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:18.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.229967899999998, 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:18.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:18.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.229945753999999, 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:18.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.229889487000001, 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.877 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:18.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:18.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:18.881 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.229873591, 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:18.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:18.886 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.886 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.886 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.886 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:18.886 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:18.886 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:18.887 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:18.887 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:18.887 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-13T09:27:18.887 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-13T09:27:18.888 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:27:18.889 INFO:teuthology.orchestra.run.smithi063.stdout:3678 trunc from 0xd8e7238 to 0x14184a0 2024-09-13T09:27:18.889 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:18.883+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:18.890 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:18.890 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:18.891 INFO:teuthology.orchestra.run.smithi063.stdout:3679 read 0xd5a586 thru 0xd5a673 (0xee bytes) 2024-09-13T09:27:18.891 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:18.892 INFO:teuthology.orchestra.run.smithi063.stdout:3680 write 0xced92af thru 0xcedb7c2 (0x2514 bytes) 2024-09-13T09:27:18.892 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:18.892 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:18.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:3682 trunc from 0xcedb7c3 to 0xd8133cd 2024-09-13T09:27:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:3689 read 0x206bb31 thru 0x2073c58 (0x8128 bytes) 2024-09-13T09:27:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:3692 read 0x62a220b thru 0x62af257 (0xd04d bytes) 2024-09-13T09:27:18.894 INFO:teuthology.orchestra.run.smithi063.stdout:3693 write 0x64bf9e9 thru 0x64c15be (0x1bd6 bytes) 2024-09-13T09:27:18.895 INFO:teuthology.orchestra.run.smithi063.stdout:3694 trunc from 0xd8133cd to 0x8b68e03 2024-09-13T09:27:18.913 INFO:teuthology.orchestra.run.smithi063.stdout:3695 read 0x1dbfa3a thru 0x1dc0898 (0xe5f bytes) 2024-09-13T09:27:18.929 INFO:teuthology.orchestra.run.smithi063.stdout:3697 write 0xad4de8f thru 0xad5c766 (0xe8d8 bytes) 2024-09-13T09:27:18.936 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:18.936 INFO:teuthology.orchestra.run.smithi063.stdout:3699 writesame 0x21cd7d thru 0x2231a8 data_size 0x6(0x642c bytes) 2024-09-13T09:27:18.943 INFO:teuthology.orchestra.run.smithi063.stdout:3702 read 0x6dfce4e thru 0x6e00159 (0x330c bytes) 2024-09-13T09:27:18.943 INFO:teuthology.orchestra.run.smithi063.stdout:3703 write 0x11e4ea0 thru 0x11ed51b (0x867c bytes) 2024-09-13T09:27:18.946 INFO:teuthology.orchestra.run.smithi063.stdout:3704 read 0xa190a16 thru 0xa1974d1 (0x6abc bytes) 2024-09-13T09:27:18.947 INFO:teuthology.orchestra.run.smithi063.stdout:3705 punch from 0x32b3bf to 0x338d35, (0xd976 bytes) 2024-09-13T09:27:18.947 INFO:teuthology.orchestra.run.smithi063.stdout:3707 clone 32 order 19 su 32768 sc 15 2024-09-13T09:27:18.988 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:18.983+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:19.036 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:19.036 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:19.036 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:19.036 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:19.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:19.089 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:19.087+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:19.190 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:19.183+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:19.197 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:19.197 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:19.197 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:19.197 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:19.197 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:19.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:19.220 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:27:19.211+0000 7f00df3a9700 -1 mon.a@0(leader).osd e89 definitely_dead 0 2024-09-13T09:27:19.291 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:19.287+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:19.313 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:19.313 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.506165800999995, 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:27:19.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:27:19.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2061 3.4 e30/21 3:25b82f25:::rbd_data.1118e3e5e25d.0000000000000001:head v 30'60, mlcod=30'58)", 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.097989916000003, 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034439050000000001, 2024-09-13T09:27:19.318 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211114+0000", 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211113+0000", 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211125+0000", 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1521e-05 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211149+0000", 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3985000000000002e-05 2024-09-13T09:27:19.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211160+0000", 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1453e-05 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211267+0000", 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106881 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7663999999999997e-05 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214470+0000", 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031651930000000002 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.214558+0000", 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8560000000000006e-05 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.097799792999993, 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.327 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2081 3.1f e30/25 3:fdfe0783:::rbd_data.1118e3e5e25d.0000000000000007:head v 30'14, mlcod=30'12)", 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.096987384000002, 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003107088, 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212117+0000", 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212116+0000", 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212125+0000", 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3749999999999992e-06 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212144+0000", 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8532000000000001e-05 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212154+0000", 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0739e-05 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212212+0000", 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7148000000000002e-05 2024-09-13T09:27:19.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.212238+0000", 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6766000000000001e-05 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215176+0000", 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002938038 2024-09-13T09:27:19.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215224+0000", 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7602000000000003e-05 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 73.908080734999999, 2024-09-13T09:27:19.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:27:19.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:27:19.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.342971349999999, 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:19.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:19.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.269432336999998, 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:19.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:19.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4692 3.4 e35/21 3:26cb438a:::rbd_object_map.11189fb5ddd6:head v 35'119, mlcod=35'115)", 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.176888969000004, 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029639739999999999, 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132215+0000", 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132214+0000", 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132226+0000", 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1987999999999999e-05 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132252+0000", 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6213000000000001e-05 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132265+0000", 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2982e-05 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132324+0000", 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8619000000000002e-05 2024-09-13T09:27:19.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.132346+0000", 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2776e-05 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135140+0000", 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002793299 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.135179+0000", 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9635999999999998e-05 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.148828393000002, 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:19.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.354 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.134570609999997, 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:19.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.665852157000003, 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:27:19.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:19.363 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.676927464000002, 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.471901416000001, 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:19.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.373563367999999, 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:27:19.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44986 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'151, mlcod=56'149)", 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.372754763000003, 2024-09-13T09:27:19.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029347290000000001, 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936350+0000", 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936349+0000", 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936356+0000", 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9410000000000001e-06 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936367+0000", 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0954999999999999e-05 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936374+0000", 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936423+0000", 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8958000000000001e-05 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.936446+0000", 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3159999999999998e-05 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939219+0000", 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027730649999999999 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939284+0000", 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4999000000000006e-05 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.298162193000003, 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:19.384 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.104079749999997, 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:19.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.988458250999997, 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:27:19.389 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:27:19.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:27:19.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.393 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 31.719716908999999, 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:19.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:106878 3.4 e69/21 3:2395b47f:::rbd_data.1118fd5aef74.0000000000000000:head v 69'245, mlcod=68'243)", 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.329741794, 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002931983, 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:19.398 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979363+0000", 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:19.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.979362+0000", 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980127+0000", 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00076519900000000004 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980141+0000", 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4419999999999999e-05 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980170+0000", 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8915e-05 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980224+0000", 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3940000000000002e-05 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.980243+0000", 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9210000000000001e-05 2024-09-13T09:27:19.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982244+0000", 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020012350000000001 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:55.982295+0000", 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0101e-05 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:19.402 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:19.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:19.404 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:19.399+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:19.505 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:19.499+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:19.522 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:19.522 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:19.522 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:19.522 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:19.533 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:19.606 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:19.603+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:19.639 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:19.639 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:19.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:19.706 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:19.703+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:19.758 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.063268850999997, 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:19.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:19.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.063246337000002, 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:19.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.063230500000003, 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.063033556000001, 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:19.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.063009231999999, 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.062992772000001, 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:19.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:19.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.062976751000001, 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:19.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:19.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.062926418000004, 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:19.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:19.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.062822842999999, 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:19.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053445046999997, 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:19.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:19.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053426383000001, 2024-09-13T09:27:19.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:19.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053409893000001, 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:19.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:19.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:19.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053394107999999, 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:19.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:19.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053377986999998, 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:19.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:19.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053362082, 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:19.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053345612999998, 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:19.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053323761000001, 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:19.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:19.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053306945999999, 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053290648999997, 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:19.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:19.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.053275048000003, 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:19.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.847 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:19.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:19.849 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:19.843+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:19.885 INFO:teuthology.orchestra.run.smithi063.stderr:marked out osd.4. 2024-09-13T09:27:19.888 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:27:19.949 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:19.947+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:19.963 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:127529 3.1d 3.8997241d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:19.885090+0000", 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.078195691999999997, 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.078236765, 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for peered", 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:19.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 127529 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885090+0000", 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885090+0000", 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885088+0000", 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885096+0000", 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3760000000000002e-06 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885109+0000", 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885123+0000", 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4103e-05 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885180+0000", 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7522999999999998e-05 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:19.885191+0000", 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0458999999999999e-05 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:19.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:19.968 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:27:19.968 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-13T09:27:19.968 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:19.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:20.050 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:20.043+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:20.128 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:20.128 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:20.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:20.151 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:20.147+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:20.245 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883401882999998, 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883386424000001, 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883370343999999, 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883354570999998, 2024-09-13T09:27:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883332670000001, 2024-09-13T09:27:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883316784000002, 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883301469000003, 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883285424, 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.8832697, 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.8832551, 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:20.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883239672000002, 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:20.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.883223786000002, 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:20.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:20.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:20.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:20.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.534096542999997, 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:20.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.533962176000003, 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:20.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.533941284999997, 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:20.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.159891680000001, 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:20.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.159014119999998, 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:20.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.158974560000001, 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9366510889999997, 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:27:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.4094378829999998, 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:27:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.337 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:20.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:20.339 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:20.335+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:20.440 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:20.435+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:20.496 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:20.496 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:20.496 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:20.497 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:20.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:20.540 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:20.535+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:20.609 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:20.610 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:20.610 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:20.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:20.641 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:20.639+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:20.726 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.163712431999997, 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.353489170000003, 2024-09-13T09:27:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.094746299000001, 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.092543960999997, 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:20.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.092147359999998, 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:20.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.091271976000002, 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.752 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.091250993000003, 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:20.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.091200481000001, 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:20.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:20.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.091170622, 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:20.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:20.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.770 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.040830939999999, 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:20.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:20.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:20.775 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.040772361999998, 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:20.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:20.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.040749726999998, 2024-09-13T09:27:20.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:20.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.040732765000001, 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:20.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:20.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.039861035000001, 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:20.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.791 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:20.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.039828401000001, 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:20.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:20.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:20.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.039808002000001, 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.039766317000002, 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:20.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.039633655999999, 2024-09-13T09:27:20.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:20.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:20.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:20.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:20.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.037283203000001, 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:20.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:20.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:20.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:20.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:20.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:20.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:20.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.037099208000001, 2024-09-13T09:27:20.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:20.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:20.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:20.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:20.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:20.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:20.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:20.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:20.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:20.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:20.828 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:20.829 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:20.829 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:20.823+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:20.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:20.945 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:20.939+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:20.988 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:20.988 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:20.988 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:20.988 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:20.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:21.031 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:21.023+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:21.056 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:21.079 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:21.102 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:21.112 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:21.112 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:21.112 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:21.112 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:21.112 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:21.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:21.125 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:21.132 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:21.127+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:21.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:21.178 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:21.214 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:21.233 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:21.227+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:21.242 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:21.245 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:21.246 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:21.265 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:21.265 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:21.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:21.334 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:21.327+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:21.417 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:21.417 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:21.429 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:21.435 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:21.427+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:21.482 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T09:27:21.534 INFO:teuthology.orchestra.run.smithi063.stdout:3708 read 0x5ca9be0 thru 0x5cb1043 (0x7464 bytes) 2024-09-13T09:27:21.537 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:21.531+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:21.542 INFO:teuthology.orchestra.run.smithi063.stdout:3709 read 0x48ff88 thru 0x49ed95 (0xee0e bytes) 2024-09-13T09:27:21.546 INFO:teuthology.orchestra.run.smithi063.stdout:3711 read 0x174e7c3 thru 0x1751e32 (0x3670 bytes) 2024-09-13T09:27:21.548 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.548 INFO:teuthology.orchestra.run.smithi063.stdout:3715 read 0x1857664 thru 0x18585c2 (0xf5f bytes) 2024-09-13T09:27:21.548 INFO:teuthology.orchestra.run.smithi063.stdout:3716 writesame 0x3ab1a6 thru 0x3ac572 data_size 0x1(0x13cd bytes) 2024-09-13T09:27:21.563 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:21.563 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:21.573 INFO:teuthology.orchestra.run.smithi063.stdout:3719 punch from 0x1b0af07 to 0x1b111e7, (0x62e0 bytes) 2024-09-13T09:27:21.575 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:21.586 INFO:teuthology.orchestra.run.smithi063.stdout:3720 write 0xc5e8364 thru 0xc5edf39 (0x5bd6 bytes) 2024-09-13T09:27:21.606 INFO:teuthology.orchestra.run.smithi063.stdout:3721 writesame 0xe333825 thru 0xe340b1d data_size 0x33(0xd2f9 bytes) 2024-09-13T09:27:21.606 INFO:teuthology.orchestra.run.smithi063.stdout:3722 write 0xc5b1269 thru 0xc5b5aaf (0x4847 bytes) 2024-09-13T09:27:21.611 INFO:teuthology.orchestra.run.smithi063.stdout:3725 writesame 0xa52bcbc thru 0xa52c5d4 data_size 0x1(0x919 bytes) 2024-09-13T09:27:21.620 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.620 INFO:teuthology.orchestra.run.smithi063.stdout:3727 write 0x9af3f42 thru 0x9af4f99 (0x1058 bytes) 2024-09-13T09:27:21.625 INFO:teuthology.orchestra.run.smithi063.stdout:3728 read 0xe08074e thru 0xe08fc16 (0xf4c9 bytes) 2024-09-13T09:27:21.625 INFO:teuthology.orchestra.run.smithi063.stdout:3729 writesame 0x510fb6f thru 0x511e4bc data_size 0x1(0xe94e bytes) 2024-09-13T09:27:21.637 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:21.631+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:21.655 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.655 INFO:teuthology.orchestra.run.smithi063.stdout:3733 read 0x797f781 thru 0x798d513 (0xdd93 bytes) 2024-09-13T09:27:21.656 INFO:teuthology.orchestra.run.smithi063.stdout:3734 write 0x2c4ad59 thru 0x2c59262 (0xe50a bytes) 2024-09-13T09:27:21.669 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.670 INFO:teuthology.orchestra.run.smithi063.stdout:3737 read 0xccca25e thru 0xccd4157 (0x9efa bytes) 2024-09-13T09:27:21.670 INFO:teuthology.orchestra.run.smithi063.stdout:3741 write 0xa71ad48 thru 0xa71f648 (0x4901 bytes) 2024-09-13T09:27:21.674 INFO:teuthology.orchestra.run.smithi063.stdout:3742 write 0xcd1babf thru 0xcd2420a (0x874c bytes) 2024-09-13T09:27:21.679 INFO:teuthology.orchestra.run.smithi063.stdout:3744 punch from 0xa10adf1 to 0xa10e8d5, (0x3ae4 bytes) 2024-09-13T09:27:21.679 INFO:teuthology.orchestra.run.smithi063.stdout:3745 write 0x37a7310 thru 0x37b25bb (0xb2ac bytes) 2024-09-13T09:27:21.692 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.692 INFO:teuthology.orchestra.run.smithi063.stdout:3747 write 0x6dd89e9 thru 0x6de32bc (0xa8d4 bytes) 2024-09-13T09:27:21.716 INFO:teuthology.orchestra.run.smithi063.stdout:3748 trunc from 0xe340b1e to 0x2c11de1 2024-09-13T09:27:21.718 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:21.718 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:21.733 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:21.772 INFO:teuthology.orchestra.run.smithi063.stdout:3749 trunc from 0x2c11de1 to 0x710181a 2024-09-13T09:27:21.772 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:21.767+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:21.782 INFO:teuthology.orchestra.run.smithi063.stdout:3750 punch from 0x231dfbc to 0x2328bd9, (0xac1d bytes) 2024-09-13T09:27:21.783 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.805 INFO:teuthology.orchestra.run.smithi063.stdout:3752 write 0xccd7b1f thru 0xcce1f93 (0xa475 bytes) 2024-09-13T09:27:21.815 INFO:teuthology.orchestra.run.smithi063.stdout:3754 read 0x2d9c07a thru 0x2da7254 (0xb1db bytes) 2024-09-13T09:27:21.815 INFO:teuthology.orchestra.run.smithi063.stdout:3755 writesame 0x5374373 thru 0x5376da2 data_size 0x5a(0x2a30 bytes) 2024-09-13T09:27:21.820 INFO:teuthology.orchestra.run.smithi063.stdout:3756 read 0xadf12e6 thru 0xadf2dea (0x1b05 bytes) 2024-09-13T09:27:21.836 INFO:teuthology.orchestra.run.smithi063.stdout:3757 trunc from 0xcce1f94 to 0x2d08bb1 2024-09-13T09:27:21.839 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:21.831+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:21.862 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.862 INFO:teuthology.orchestra.run.smithi063.stdout:3762 punch from 0x170b1be to 0x170c67b, (0x14bd bytes) 2024-09-13T09:27:21.862 INFO:teuthology.orchestra.run.smithi063.stdout:3763 punch from 0x178064f to 0x1787bfc, (0x75ad bytes) 2024-09-13T09:27:21.862 INFO:teuthology.orchestra.run.smithi063.stdout:3765 read 0xb6d4c6 thru 0xb71eb9 (0x49f4 bytes) 2024-09-13T09:27:21.863 INFO:teuthology.orchestra.run.smithi063.stdout:3767 read 0x26f4709 thru 0x26f9865 (0x515d bytes) 2024-09-13T09:27:21.864 INFO:teuthology.orchestra.run.smithi063.stdout:3768 punch from 0xbe30e0 to 0xbe57de, (0x26fe bytes) 2024-09-13T09:27:21.869 INFO:teuthology.orchestra.run.smithi063.stdout:3769 trunc from 0x2d08bb1 to 0xb1cd7e 2024-09-13T09:27:21.899 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:27:21.913 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:21.913 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:21.925 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:21.930 INFO:teuthology.orchestra.run.smithi063.stdout:3770 read 0x3f3523 thru 0x3fb4c7 (0x7fa5 bytes) 2024-09-13T09:27:21.940 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:21.935+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:21.958 INFO:teuthology.orchestra.run.smithi063.stdout:3771 write 0x4180d9f thru 0x41843fc (0x365e bytes) 2024-09-13T09:27:21.964 INFO:teuthology.orchestra.run.smithi063.stdout:3772 read 0x2722e5a thru 0x272b2e8 (0x848f bytes) 2024-09-13T09:27:21.964 INFO:teuthology.orchestra.run.smithi063.stdout:3773 punch from 0x2167771 to 0x2173927, (0xc1b6 bytes) 2024-09-13T09:27:21.981 INFO:teuthology.orchestra.run.smithi063.stdout:3774 writesame 0x8225bb9 thru 0x8231e60 data_size 0x4(0xc2a8 bytes) 2024-09-13T09:27:21.987 INFO:teuthology.orchestra.run.smithi063.stdout:3775 trunc from 0x8231e61 to 0xb7eaa7d 2024-09-13T09:27:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:3778 read 0x9805ec0 thru 0x9807f79 (0x20ba bytes) 2024-09-13T09:27:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:3780 read 0xab51b4c thru 0xab5a710 (0x8bc5 bytes) 2024-09-13T09:27:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:3781 punch from 0x1f54bb to 0x2048a4, (0xf3e9 bytes) 2024-09-13T09:27:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:21.998 INFO:teuthology.orchestra.run.smithi063.stdout:3783 read 0x41ea5c thru 0x4277f4 (0x8d99 bytes) 2024-09-13T09:27:21.999 INFO:teuthology.orchestra.run.smithi063.stdout:3784 write 0x4a3c598 thru 0x4a45b6c (0x95d5 bytes) 2024-09-13T09:27:22.007 INFO:teuthology.orchestra.run.smithi063.stdout:3787 punch from 0xa25e899 to 0xa26335b, (0x4ac2 bytes) 2024-09-13T09:27:22.007 INFO:teuthology.orchestra.run.smithi063.stdout:3788 punch from 0x180b153 to 0x180c1cc, (0x1079 bytes) 2024-09-13T09:27:22.008 INFO:teuthology.orchestra.run.smithi063.stdout:3789 punch from 0x11fc4ea to 0x11fe13b, (0x1c51 bytes) 2024-09-13T09:27:22.008 INFO:teuthology.orchestra.run.smithi063.stdout:3790 read 0x2ad238a thru 0x2ae0b3a (0xe7b1 bytes) 2024-09-13T09:27:22.008 INFO:teuthology.orchestra.run.smithi063.stdout:3791 write 0x283eef9 thru 0x2840463 (0x156b bytes) 2024-09-13T09:27:22.013 INFO:teuthology.orchestra.run.smithi063.stdout:3792 write 0x93ad578 thru 0x93b7e51 (0xa8da bytes) 2024-09-13T09:27:22.019 INFO:teuthology.orchestra.run.smithi063.stdout:3793 read 0x15220bd thru 0x15265bd (0x4501 bytes) 2024-09-13T09:27:22.019 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:22.019 INFO:teuthology.orchestra.run.smithi063.stdout:3797 read 0x9d6301 thru 0x9d9160 (0x2e60 bytes) 2024-09-13T09:27:22.029 INFO:teuthology.orchestra.run.smithi063.stdout:3799 write 0xbd6efa3 thru 0xbd7a59e (0xb5fc bytes) 2024-09-13T09:27:22.041 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:22.035+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:22.045 INFO:teuthology.orchestra.run.smithi063.stdout:3800 write 0x7d56cdc thru 0x7d59584 (0x28a9 bytes) 2024-09-13T09:27:22.051 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:22.051 INFO:teuthology.orchestra.run.smithi063.stdout:3802 punch from 0x236ec38 to 0x237383a, (0x4c02 bytes) 2024-09-13T09:27:22.052 INFO:teuthology.orchestra.run.smithi063.stdout:3803 read 0xa6109a0 thru 0xa61a531 (0x9b92 bytes) 2024-09-13T09:27:22.052 INFO:teuthology.orchestra.run.smithi063.stdout:3805 read 0xad5590c thru 0xad6022b (0xa920 bytes) 2024-09-13T09:27:22.052 INFO:teuthology.orchestra.run.smithi063.stdout:3806 trunc from 0xbd7a59f to 0xc5b0f06 2024-09-13T09:27:22.054 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.054 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.062 INFO:teuthology.orchestra.run.smithi063.stdout:3808 read 0x4337611 thru 0x433c006 (0x49f6 bytes) 2024-09-13T09:27:22.065 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:22.065 INFO:teuthology.orchestra.run.smithi063.stdout:3809 trunc from 0xc5b0f06 to 0xb0191b4 2024-09-13T09:27:22.081 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:22.081 INFO:teuthology.orchestra.run.smithi063.stdout:3811 read 0x3186308 thru 0x318647a (0x173 bytes) 2024-09-13T09:27:22.096 INFO:teuthology.orchestra.run.smithi063.stdout:3812 write 0xdc3e9e6 thru 0xdc4abb1 (0xc1cc bytes) 2024-09-13T09:27:22.107 INFO:teuthology.orchestra.run.smithi063.stdout:3813 punch from 0x2e598c2 to 0x2e5e999, (0x50d7 bytes) 2024-09-13T09:27:22.107 INFO:teuthology.orchestra.run.smithi063.stdout:3814 writesame 0x2a2247e thru 0x2a2e8c3 data_size 0x25(0xc446 bytes) 2024-09-13T09:27:22.115 INFO:teuthology.orchestra.run.smithi063.stdout:3817 write 0x253ed03 thru 0x254c70f (0xda0d bytes) 2024-09-13T09:27:22.127 INFO:teuthology.orchestra.run.smithi063.stdout:3819 read 0x6261fed thru 0x62676ee (0x5702 bytes) 2024-09-13T09:27:22.127 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:22.127 INFO:teuthology.orchestra.run.smithi063.stdout:3821 punch from 0x6f4befd to 0x6f4e3a3, (0x24a6 bytes) 2024-09-13T09:27:22.127 INFO:teuthology.orchestra.run.smithi063.stdout:3822 write 0x3e6bab7 thru 0x3e76333 (0xa87d bytes) 2024-09-13T09:27:22.136 INFO:teuthology.orchestra.run.smithi063.stdout:3823 write 0xdb44c8a thru 0xdb45394 (0x70b bytes) 2024-09-13T09:27:22.141 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:22.135+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:22.142 INFO:teuthology.orchestra.run.smithi063.stdout:3824 write 0x14470e4 thru 0x1452925 (0xb842 bytes) 2024-09-13T09:27:22.166 INFO:teuthology.orchestra.run.smithi063.stdout:3825 trunc from 0xdc4abb2 to 0x69adcb6 2024-09-13T09:27:22.209 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.210 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.224 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:22.242 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:22.239+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:22.343 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:22.339+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:22.389 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.389 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.400 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:22.445 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:22.439+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:22.513 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.513 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.523 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:22.546 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:22.543+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:22.627 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.627 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:22.647 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:22.643+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:22.748 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.749 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.749 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:22.743+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:22.757 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:22.850 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:22.843+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:22.867 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:22.868 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:22.878 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:22.951 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:22.947+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:23.034 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:23.034 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:23.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:23.090 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:23.087+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:23.152 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:23.147+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:23.159 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:23.159 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:23.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:23.175 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:27:23.254 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:23.251+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:23.260 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:23.271 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:23.272 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:23.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:23.355 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:23.347+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:23.390 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:23.391 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:23.400 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:23.418 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:23.441 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:23.455 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:23.447+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:23.463 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:23.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:23.505 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:23.513 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:23.513 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:23.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:23.525 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:23.547 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:23.556 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:23.547+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:23.570 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:23.573 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:23.630 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:23.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:23.630 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:23.630 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:23.630 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:23.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:23.656 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:23.651+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:23.744 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382433235999997, 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382417777000001, 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382401696999999, 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382385923999998, 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382364023000001, 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382348137000001, 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382332822000002, 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382316777, 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382301052999999, 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382286452999999, 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382271025000001, 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.382255139000002, 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.033127896000003, 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.032993529000002, 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.032972637999997, 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.658923033000001, 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.658045473000001, 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.658005913, 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.435682441999999, 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:27:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:27:23.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:23.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9084692360000002, 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:27:23.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:23.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:27:23.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:23.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:23.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:23.856 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:23.859 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:23.851+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:23.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:23.958 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:23.955+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:23.970 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:23.970 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:23.970 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:23.970 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:23.980 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:24.059 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:24.051+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:24.131 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:24.132 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:24.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:24.160 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:24.155+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:24.233 INFO:teuthology.orchestra.run.smithi063.stdout:3826 write 0xbf3bf0e thru 0xbf41f58 (0x604b bytes) 2024-09-13T09:27:24.243 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.243 INFO:teuthology.orchestra.run.smithi063.stdout:3831 write 0x77d4dd7 thru 0x77d8f1f (0x4149 bytes) 2024-09-13T09:27:24.250 INFO:teuthology.orchestra.run.smithi063.stdout:3832 read 0x451ab23 thru 0x451c603 (0x1ae1 bytes) 2024-09-13T09:27:24.261 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:24.255+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.700155553000002, 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.889932291000001, 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.631189419999998, 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.628987082000002, 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.628590481000003, 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.627715096999999, 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.627694114000001, 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.627643601999999, 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.627613742999998, 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.577274060999997, 2024-09-13T09:27:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.577215483000003, 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.577192848000003, 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.577175885999999, 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.576304155999999, 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.576271521999999, 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.576251122999999, 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.576209437999999, 2024-09-13T09:27:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:24.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.576076776999997, 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.362 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.573726323999999, 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:24.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.573542328999999, 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:24.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:24.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.373 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:24.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:24.376 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:24.371+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:24.377 INFO:teuthology.orchestra.run.smithi063.stdout:3833 write 0xe3cf90d thru 0xe3de7f8 (0xeeec bytes) 2024-09-13T09:27:24.377 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.377 INFO:teuthology.orchestra.run.smithi063.stdout:3835 write 0xdbbdce7 thru 0xdbc6307 (0x8621 bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3836 read 0xd72fd67 thru 0xd73e3d8 (0xe672 bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3837 write 0x1d9e315 thru 0x1daa90e (0xc5fa bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3839 read 0x32388da thru 0x3238fef (0x716 bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3841 read 0xaff6f60 thru 0xb002a4e (0xbaef bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3842 read 0x9b691bd thru 0x9b6975b (0x59f bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3845 read 0xae6c6e6 thru 0xae7710f (0xaa2a bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3848 writesame 0x2fce985 thru 0x2fd8778 data_size 0x16(0x9df4 bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3849 trunc from 0xe3de7f9 to 0xcf86464 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3850 read 0xa610a7e thru 0xa6157fa (0x4d7d bytes) 2024-09-13T09:27:24.378 INFO:teuthology.orchestra.run.smithi063.stdout:3851 write 0xc35da15 thru 0xc364c5a (0x7246 bytes) 2024-09-13T09:27:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:3854 write 0x7d31bb5 thru 0x7d31ee3 (0x32f bytes) 2024-09-13T09:27:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:3855 write 0x858bfd thru 0x859981 (0xd85 bytes) 2024-09-13T09:27:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:3856 writesame 0x2483447 thru 0x248adc8 data_size 0x67(0x7982 bytes) 2024-09-13T09:27:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:3857 punch from 0x625f55 to 0x633016, (0xd0c1 bytes) 2024-09-13T09:27:24.379 INFO:teuthology.orchestra.run.smithi063.stdout:3859 trunc from 0xcf86464 to 0x78254a9 2024-09-13T09:27:24.411 INFO:teuthology.orchestra.run.smithi063.stdout:3860 writesame 0xaba0f38 thru 0xabaf2a3 data_size 0x29(0xe36c bytes) 2024-09-13T09:27:24.411 INFO:teuthology.orchestra.run.smithi063.stdout:3862 writesame 0x6147cfc thru 0x614f9a3 data_size 0x4(0x7ca8 bytes) 2024-09-13T09:27:24.442 INFO:teuthology.orchestra.run.smithi063.stdout:3863 trunc from 0xabaf2a4 to 0x5d49c5c 2024-09-13T09:27:24.468 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.475 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:24.467+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:24.491 INFO:teuthology.orchestra.run.smithi063.stdout:3868 writesame 0xa7e299f thru 0xa7e50cd data_size 0x1(0x272f bytes) 2024-09-13T09:27:24.491 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.491 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.492 INFO:teuthology.orchestra.run.smithi063.stdout:3871 punch from 0xa1ccf05 to 0xa1d5bd6, (0x8cd1 bytes) 2024-09-13T09:27:24.492 INFO:teuthology.orchestra.run.smithi063.stdout:3872 punch from 0x91a9499 to 0x91b6352, (0xceb9 bytes) 2024-09-13T09:27:24.492 INFO:teuthology.orchestra.run.smithi063.stdout:3873 read 0x7a8480a thru 0x7a8fa82 (0xb279 bytes) 2024-09-13T09:27:24.492 INFO:teuthology.orchestra.run.smithi063.stdout:3874 write 0x4f04126 thru 0x4f0bc96 (0x7b71 bytes) 2024-09-13T09:27:24.502 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.502 INFO:teuthology.orchestra.run.smithi063.stdout:3877 punch from 0x41f446 to 0x422690, (0x324a bytes) 2024-09-13T09:27:24.523 INFO:teuthology.orchestra.run.smithi063.stdout:3879 trunc from 0xa7e50ce to 0xe3c32b 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:128687 3.15 3:ab98b42a:::rbd_data.11181500dc3e.000000000000001b:head [stat out=16b,truncate 458752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:24.539391+0000", 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0022077249999999998, 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022281929999999998, 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 128687 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539391+0000", 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539391+0000", 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539390+0000", 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539394+0000", 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.481e-06 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539403+0000", 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1769999999999998e-06 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539409+0000", 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.404e-06 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539766+0000", 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035666099999999998 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539827+0000", 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0934999999999998e-05 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.539927+0000", 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9894000000000003e-05 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:24.541156+0000", 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001229883 2024-09-13T09:27:24.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.547 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:27:24.547 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-13T09:27:24.547 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:24.548 INFO:teuthology.orchestra.run.smithi063.stdout:3882 read 0x3c4c67 thru 0x3ca14a (0x54e4 bytes) 2024-09-13T09:27:24.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:24.568 INFO:teuthology.orchestra.run.smithi063.stdout:3883 write 0x4beb408 thru 0x4bef01f (0x3c18 bytes) 2024-09-13T09:27:24.576 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:24.571+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:24.581 INFO:teuthology.orchestra.run.smithi063.stdout:3884 write 0x5237393 thru 0x524157d (0xa1eb bytes) 2024-09-13T09:27:24.591 INFO:teuthology.orchestra.run.smithi063.stdout:3886 read 0x218481 thru 0x2251a8 (0xcd28 bytes) 2024-09-13T09:27:24.593 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.593 INFO:teuthology.orchestra.run.smithi063.stdout:3889 punch from 0x2a89ccb to 0x2a98333, (0xe668 bytes) 2024-09-13T09:27:24.593 INFO:teuthology.orchestra.run.smithi063.stdout:3890 read 0x2402658 thru 0x240dc07 (0xb5b0 bytes) 2024-09-13T09:27:24.593 INFO:teuthology.orchestra.run.smithi063.stdout:3891 read 0x13d0f48 thru 0x13dbef7 (0xafb0 bytes) 2024-09-13T09:27:24.598 INFO:teuthology.orchestra.run.smithi063.stdout:3894 trunc from 0x524157e to 0x306fa75 2024-09-13T09:27:24.623 INFO:teuthology.orchestra.run.smithi063.stdout:3895 read 0x2f3a938 thru 0x2f47c59 (0xd322 bytes) 2024-09-13T09:27:24.623 INFO:teuthology.orchestra.run.smithi063.stdout:3896 write 0xac4dc thru 0xbb86a (0xf38f bytes) 2024-09-13T09:27:24.648 INFO:teuthology.orchestra.run.smithi063.stdout:3897 trunc from 0x306fa75 to 0xc5c24f8 2024-09-13T09:27:24.658 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:24.658 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:24.658 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:24.658 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:24.658 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:24.660 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.660 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.660 INFO:teuthology.orchestra.run.smithi063.stdout:3900 write 0x579b25c thru 0x579ce2a (0x1bcf bytes) 2024-09-13T09:27:24.666 INFO:teuthology.orchestra.run.smithi063.stdout:3902 writesame 0x2e77664 thru 0x2e79a9d data_size 0x1(0x243a bytes) 2024-09-13T09:27:24.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:24.677 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:24.671+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:24.684 INFO:teuthology.orchestra.run.smithi063.stdout:3903 trunc from 0xc5c24f8 to 0x829a911 2024-09-13T09:27:24.696 INFO:teuthology.orchestra.run.smithi063.stdout:3904 read 0x6ba39bc thru 0x6bb2791 (0xedd6 bytes) 2024-09-13T09:27:24.696 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.697 INFO:teuthology.orchestra.run.smithi063.stdout:3907 write 0x4a4cd39 thru 0x4a568e0 (0x9ba8 bytes) 2024-09-13T09:27:24.705 INFO:teuthology.orchestra.run.smithi063.stdout:3908 read 0x42b390 thru 0x42ee73 (0x3ae4 bytes) 2024-09-13T09:27:24.706 INFO:teuthology.orchestra.run.smithi063.stdout:3909 read 0x272fe73 thru 0x273fc7f (0xfe0d bytes) 2024-09-13T09:27:24.729 INFO:teuthology.orchestra.run.smithi063.stdout:3910 write 0xa2fe884 thru 0xa30bbbe (0xd33b bytes) 2024-09-13T09:27:24.740 INFO:teuthology.orchestra.run.smithi063.stdout:3912 write 0x7b4a571 thru 0x7b59ab6 (0xf546 bytes) 2024-09-13T09:27:24.748 INFO:teuthology.orchestra.run.smithi063.stdout:3914 write 0x75dc937 thru 0x75e4f2c (0x85f6 bytes) 2024-09-13T09:27:24.763 INFO:teuthology.orchestra.run.smithi063.stdout:3915 trunc from 0xa30bbbf to 0xe49add8 2024-09-13T09:27:24.778 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:24.771+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:24.779 INFO:teuthology.orchestra.run.smithi063.stdout:3917 trunc from 0xe49add8 to 0xcdfe970 2024-09-13T09:27:24.792 INFO:teuthology.orchestra.run.smithi063.stdout:3918 writesame 0x450690 thru 0x458252 data_size 0x3b(0x7bc3 bytes) 2024-09-13T09:27:24.809 INFO:teuthology.orchestra.run.smithi063.stdout:3919 read 0x9e2d4cc thru 0x9e35922 (0x8457 bytes) 2024-09-13T09:27:24.809 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.809 INFO:teuthology.orchestra.run.smithi063.stdout:3922 read 0x691a20e thru 0x692185f (0x7652 bytes) 2024-09-13T09:27:24.809 INFO:teuthology.orchestra.run.smithi063.stdout:3924 read 0x4572c96 thru 0x45796a1 (0x6a0c bytes) 2024-09-13T09:27:24.809 INFO:teuthology.orchestra.run.smithi063.stdout:3925 read 0x2dccd9a thru 0x2dd77b7 (0xaa1e bytes) 2024-09-13T09:27:24.812 INFO:teuthology.orchestra.run.smithi063.stdout:3926 trunc from 0xcdfe970 to 0xe750cd9 2024-09-13T09:27:24.825 INFO:teuthology.orchestra.run.smithi063.stdout:3927 writesame 0xb307019 thru 0xb30a6ea data_size 0x3(0x36d2 bytes) 2024-09-13T09:27:24.826 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:24.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.86265070499999, 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:24.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:24.831 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.749430961000002, 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:24.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.749032946, 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:24.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073926069999999, 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073911138, 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:24.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:24.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073878032000003, 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:24.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073861739000002, 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:24.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073845624, 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:24.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073814583000001, 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:24.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.073782960000003, 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:24.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.062889648000002, 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:27:24.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:27:24.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.062869311, 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:27:24.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:27:24.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:24.884 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060834899, 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:24.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060817843000002, 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.889 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:24.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060801675, 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:24.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:24.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.897 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060588013, 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:24.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:24.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060568701999998, 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:24.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:24.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060552309000002, 2024-09-13T09:27:24.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:24.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060536646999999, 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:24.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.060510293, 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:24.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:24.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:24.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:24.919 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:24.920 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:24.920 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:24.920 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:27:24.920 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:27:24.922 INFO:teuthology.orchestra.run.smithi063.stdout:3928 writesame 0x40e1ea thru 0x41de77 data_size 0x1(0xfc8e bytes) 2024-09-13T09:27:24.922 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:24.922 INFO:teuthology.orchestra.run.smithi063.stdout:3930 trunc from 0xe750cd9 to 0x2597e4a 2024-09-13T09:27:24.922 INFO:teuthology.orchestra.run.smithi063.stdout:3931 read 0x17dbe02 thru 0x17dd70e (0x190d bytes) 2024-09-13T09:27:24.922 INFO:teuthology.orchestra.run.smithi063.stdout:3932 trunc from 0x2597e4a to 0x38a23fb 2024-09-13T09:27:24.923 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:24.923 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:24.915+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:24.957 INFO:teuthology.orchestra.run.smithi063.stdout:3933 write 0xe16282e thru 0xe17130a (0xeadd bytes) 2024-09-13T09:27:24.967 INFO:teuthology.orchestra.run.smithi063.stdout:3934 write 0x59b2662 thru 0x59ba0fc (0x7a9b bytes) 2024-09-13T09:27:24.985 INFO:teuthology.orchestra.run.smithi063.stdout:3935 trunc from 0xe17130b to 0x9126f3e 2024-09-13T09:27:25.005 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.005 INFO:teuthology.orchestra.run.smithi063.stdout:3937 read 0x84a93b5 thru 0x84b8d67 (0xf9b3 bytes) 2024-09-13T09:27:25.005 INFO:teuthology.orchestra.run.smithi063.stdout:3938 punch from 0x35e1835 to 0x35e1f3f, (0x70a bytes) 2024-09-13T09:27:25.005 INFO:teuthology.orchestra.run.smithi063.stdout:3939 write 0x10438c3 thru 0x10457ec (0x1f2a bytes) 2024-09-13T09:27:25.011 INFO:teuthology.orchestra.run.smithi063.stdout:3940 write 0x1c70353 thru 0x1c7e685 (0xe333 bytes) 2024-09-13T09:27:25.019 INFO:teuthology.orchestra.run.smithi063.stdout:3941 read 0x89dc548 thru 0x89e705f (0xab18 bytes) 2024-09-13T09:27:25.021 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:25.015+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:25.024 INFO:teuthology.orchestra.run.smithi063.stdout:3942 trunc from 0x9126f3e to 0x5dcef97 2024-09-13T09:27:25.040 INFO:teuthology.orchestra.run.smithi063.stdout:3943 read 0x1461249 thru 0x146f63d (0xe3f5 bytes) 2024-09-13T09:27:25.040 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.040 INFO:teuthology.orchestra.run.smithi063.stdout:3945 writesame 0x48b30dc thru 0x48bb1e8 data_size 0x1(0x810d bytes) 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:129046 3.1 e95/93 3:87b12a70:::rbd_object_map.11181500dc3e:head v 95'791, mlcod=95'789)", 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.039838+0000", 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.00017713900000000001, 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000195043, 2024-09-13T09:27:25.041 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039838+0000", 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039838+0000", 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039837+0000", 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039845+0000", 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.092e-06 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039858+0000", 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3671e-05 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039865+0000", 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8109999999999997e-06 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039925+0000", 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0340000000000002e-05 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.039960+0000", 2024-09-13T09:27:25.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4888999999999997e-05 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 1 2024-09-13T09:27:25.045 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:25.055 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:25.070 INFO:teuthology.orchestra.run.smithi063.stdout:3946 writesame 0x6dd19ba thru 0x6dd63a6 data_size 0x5(0x49ed bytes) 2024-09-13T09:27:25.071 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.071 INFO:teuthology.orchestra.run.smithi063.stdout:3949 write 0x4ab658 thru 0x4b54f3 (0x9e9c bytes) 2024-09-13T09:27:25.090 INFO:teuthology.orchestra.run.smithi063.stdout:3950 write 0x75e26c8 thru 0x75ea979 (0x82b2 bytes) 2024-09-13T09:27:25.097 INFO:teuthology.orchestra.run.smithi063.stdout:3952 punch from 0x1e3f2ac to 0x1e44187, (0x4edb bytes) 2024-09-13T09:27:25.103 INFO:teuthology.orchestra.run.smithi063.stdout:3953 trunc from 0x75ea97a to 0xb0a6db5 2024-09-13T09:27:25.122 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:25.119+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:25.127 INFO:teuthology.orchestra.run.smithi063.stdout:3954 write 0xb597b21 thru 0xb5a1bb0 (0xa090 bytes) 2024-09-13T09:27:25.142 INFO:teuthology.orchestra.run.smithi063.stdout:3956 write 0xa6a73bc thru 0xa6ac400 (0x5045 bytes) 2024-09-13T09:27:25.167 INFO:teuthology.orchestra.run.smithi063.stdout:3957 write 0xec20c15 thru 0xec28fa2 (0x838e bytes) 2024-09-13T09:27:25.173 INFO:teuthology.orchestra.run.smithi063.stdout:3958 writesame 0x24a86c9 thru 0x24b1489 data_size 0x1(0x8dc1 bytes) 2024-09-13T09:27:25.182 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:25.182 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:25.182 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:25.182 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:25.182 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:25.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:27:25.193 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:25.199 INFO:teuthology.orchestra.run.smithi063.stdout:3960 read 0x4a28305 thru 0x4a2b323 (0x301f bytes) 2024-09-13T09:27:25.209 INFO:teuthology.orchestra.run.smithi063.stdout:3961 trunc from 0xec28fa3 to 0x8b425f9 2024-09-13T09:27:25.223 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:25.215+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:25.235 INFO:teuthology.orchestra.run.smithi063.stdout:3962 write 0xf8a5d1 thru 0xf8e17a (0x3baa bytes) 2024-09-13T09:27:25.241 INFO:teuthology.orchestra.run.smithi063.stdout:3963 writesame 0x704724 thru 0x70cd11 data_size 0x4f(0x85ee bytes) 2024-09-13T09:27:25.245 INFO:teuthology.orchestra.run.smithi063.stdout:3965 writesame 0x469bb76 thru 0x46ab43d data_size 0x4c(0xf8c8 bytes) 2024-09-13T09:27:25.257 INFO:teuthology.orchestra.run.smithi063.stdout:3966 read 0x25ad021 thru 0x25b4dfd (0x7ddd bytes) 2024-09-13T09:27:25.257 INFO:teuthology.orchestra.run.smithi063.stdout:3967 punch from 0xfcfb21 to 0xfd95ba, (0x9a99 bytes) 2024-09-13T09:27:25.257 INFO:teuthology.orchestra.run.smithi063.stdout:3968 punch from 0x7cef00d to 0x7cfc12e, (0xd121 bytes) 2024-09-13T09:27:25.257 INFO:teuthology.orchestra.run.smithi063.stdout:3969 punch from 0x20f108e to 0x20fd335, (0xc2a7 bytes) 2024-09-13T09:27:25.257 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.278 INFO:teuthology.orchestra.run.smithi063.stdout:3972 writesame 0xe04b315 thru 0xe04e703 data_size 0x1(0x33ef bytes) 2024-09-13T09:27:25.278 INFO:teuthology.orchestra.run.smithi063.stdout:3973 punch from 0xd673b7c to 0xd6779c3, (0x3e47 bytes) 2024-09-13T09:27:25.281 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:25.281 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:25.281 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 95, 2024-09-13T09:27:25.281 INFO:teuthology.orchestra.run.smithi063.stdout: "fsid": "eb7f03c3-9691-4766-8999-6f4f526cda71", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "created": "2024-09-13T09:25:30.379173+0000", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "modified": "2024-09-13T09:27:24.174472+0000", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "last_up_change": "2024-09-13T09:27:01.686116+0000", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "last_in_change": "2024-09-13T09:27:19.857554+0000", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_num": 5804032, 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_set": [ 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "nodeep-scrub", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "pglog_hardlimit", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "purged_snapdirs", 2024-09-13T09:27:25.282 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_deletes", 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "sortbitwise" 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_version": 6, 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_snapshot": "", 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_max": 3, 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd": 8, 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:27:25.283 INFO:teuthology.orchestra.run.smithi063.stdout: "min_compat_client": "jewel", 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "require_osd_release": "reef", 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_crimson": false, 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "pools": [ 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 1, 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "rbd", 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:41.357433+0000", 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:25.284 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:25.285 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:25.285 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:25.285 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:25.285 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:25.285 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:25.285 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 8, 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 8, 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 8, 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 8, 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 8, 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:25.286 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:25.287 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 3, 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 19, 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:25.288 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:25.289 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:25.290 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 1.9900000095367432, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 1.9900000095367432, 2024-09-13T09:27:25.291 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 1.75, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 1.75, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 2, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": ".mgr", 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:42.578616+0000", 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-09-13T09:27:25.292 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool", 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 1, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 1, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 1, 2024-09-13T09:27:25.293 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 1, 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 1, 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:25.294 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 0, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 0, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:25.295 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:25.296 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "options": { 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_max": 32, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_min": 1 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": {} 2024-09-13T09:27:25.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 8, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 8, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 0.25, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 3, 2024-09-13T09:27:25.298 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "pool_client.0", 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:51.228779+0000", 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:25.299 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 32, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 32, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 32, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 32, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 32, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "90", 2024-09-13T09:27:25.300 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 35, 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 90, 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:25.301 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:25.302 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:25.303 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 1.7400000095367432, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 1.7400000095367432, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:27:25.304 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "osds": [ 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "54ea2bd8-55b0-4ded-8a16-041261c5ba5a", 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.305 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 93, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6816", 2024-09-13T09:27:25.306 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6817", 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6818", 2024-09-13T09:27:25.307 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6819", 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.308 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6822", 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6823", 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.309 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6820", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6821", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6817/2906107514", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6819/2906107514", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6823/2906107514", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6821/2906107514", 2024-09-13T09:27:25.310 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "0c95aee9-f871-4f6f-bbbd-1206db5d331c", 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.311 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 93, 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.312 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6824", 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6825", 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.314 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6826", 2024-09-13T09:27:25.314 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6827", 2024-09-13T09:27:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6830", 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6831", 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6828", 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6829", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6825/3622854518", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6827/3622854518", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6831/3622854518", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6829/3622854518", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.318 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "f8253db2-d4da-46b5-95cf-cd5c9ebd7328", 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.319 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 14, 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 93, 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6800", 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6801", 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.321 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6802", 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6803", 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.322 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6806", 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6807", 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6804", 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6805", 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6801/2105659441", 2024-09-13T09:27:25.325 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6803/2105659441", 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6807/2105659441", 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6805/2105659441", 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "67c11a52-baeb-4ea8-89ea-26018ffdc35a", 2024-09-13T09:27:25.326 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 15, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 46, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 74, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 48, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.327 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6808", 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6809", 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.328 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6810", 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6811", 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6814", 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6815", 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.330 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6812", 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6813", 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:25.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6809/4293004354", 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6811/4293004354", 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6815/4293004354", 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6813/4293004354", 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.332 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "1cec9702-fa09-4d7c-b111-7d10b1950cfe", 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 0, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 0, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 60, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6800", 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6801", 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.334 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6802", 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6803", 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.335 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6806", 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6807", 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.336 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6804", 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6805", 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6801/1789187713", 2024-09-13T09:27:25.337 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6803/1789187713", 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1789187713", 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1789187713", 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "7c0b908a-65c9-426c-867c-cc8ed65084b3", 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.338 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 93, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6809", 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6811", 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6813", 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6815", 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6821", 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6823", 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6817", 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6819", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6811/3410135021", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6815/3410135021", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3410135021", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6819/3410135021", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.343 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "78d5990a-4535-4110-ba04-88f01ab9fe28", 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:25.344 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 93, 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6808", 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6810", 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6812", 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6814", 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6820", 2024-09-13T09:27:25.347 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6822", 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6816", 2024-09-13T09:27:25.348 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6818", 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6810/4152410871", 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6814/4152410871", 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6822/4152410871", 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6818/4152410871", 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "53f8511c-729f-4423-8eac-813c01f8f72b", 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:25.351 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 94, 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6824", 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6825", 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.353 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6826", 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6827", 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.354 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6830", 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6831", 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6828", 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6829", 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:25.356 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6825/2923799929", 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6827/2923799929", 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2923799929", 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2923799929", 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:25.357 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_xinfo": [ 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.267895+0000", 2024-09-13T09:27:25.358 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.359 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.161313+0000", 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.447964+0000", 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.361 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.021320+0000", 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 48 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 65536, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.883961+0000", 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.362 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.938146+0000", 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.363 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.627237+0000", 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.887523+0000", 2024-09-13T09:27:25.364 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap": [], 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_items": [], 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_primaries": [], 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_temp": [], 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_temp": [], 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklist": {}, 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "range_blocklist": {}, 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profiles": { 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "default": { 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-failure-domain": "osd", 2024-09-13T09:27:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: "k": "2", 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "m": "1", 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "plugin": "jerasure", 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "technique": "reed_sol_van" 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps_queue": [], 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "new_removed_snaps": [], 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "new_purged_snaps": [], 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_node_flags": {}, 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "device_class_flags": {}, 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode": { 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_enabled": false, 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_bucket_count": 0, 2024-09-13T09:27:25.366 INFO:teuthology.orchestra.run.smithi063.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:27:25.367 INFO:teuthology.orchestra.run.smithi063.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:27:25.367 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_bucket": 0 2024-09-13T09:27:25.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.367 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:25.368 INFO:teuthology.orchestra.run.smithi063.stdout:3974 trunc from 0xe04e704 to 0x94bab7a 2024-09-13T09:27:25.369 INFO:tasks.thrashosds.thrasher:j is {'epoch': 95, 'fsid': 'eb7f03c3-9691-4766-8999-6f4f526cda71', 'created': '2024-09-13T09:25:30.379173+0000', 'modified': '2024-09-13T09:27:24.174472+0000', 'last_up_change': '2024-09-13T09:27:01.686116+0000', 'last_in_change': '2024-09-13T09:27:19.857554+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:25:41.357433+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:25:42.578616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:25:51.228779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 90, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '54ea2bd8-55b0-4ded-8a16-041261c5ba5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6816', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6817', 'nonce': 2906107514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6818', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6819', 'nonce': 2906107514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6822', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6823', 'nonce': 2906107514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6820', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6821', 'nonce': 2906107514}]}, 'public_addr': '172.21.15.63:6817/2906107514', 'cluster_addr': '172.21.15.63:6819/2906107514', 'heartbeat_back_addr': '172.21.15.63:6823/2906107514', 'heartbeat_front_addr': '172.21.15.63:6821/2906107514', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0c95aee9-f871-4f6f-bbbd-1206db5d331c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6824', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6825', 'nonce': 3622854518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6826', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6827', 'nonce': 3622854518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6830', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6831', 'nonce': 3622854518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6828', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6829', 'nonce': 3622854518}]}, 'public_addr': '172.21.15.63:6825/3622854518', 'cluster_addr': '172.21.15.63:6827/3622854518', 'heartbeat_back_addr': '172.21.15.63:6831/3622854518', 'heartbeat_front_addr': '172.21.15.63:6829/3622854518', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f8253db2-d4da-46b5-95cf-cd5c9ebd7328', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6800', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6801', 'nonce': 2105659441}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6802', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6803', 'nonce': 2105659441}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6806', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6807', 'nonce': 2105659441}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6804', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6805', 'nonce': 2105659441}]}, 'public_addr': '172.21.15.63:6801/2105659441', 'cluster_addr': '172.21.15.63:6803/2105659441', 'heartbeat_back_addr': '172.21.15.63:6807/2105659441', 'heartbeat_front_addr': '172.21.15.63:6805/2105659441', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '67c11a52-baeb-4ea8-89ea-26018ffdc35a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 46, 'up_from': 74, 'up_thru': 77, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6808', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6809', 'nonce': 4293004354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6810', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6811', 'nonce': 4293004354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6814', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6815', 'nonce': 4293004354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6812', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6813', 'nonce': 4293004354}]}, 'public_addr': '172.21.15.63:6809/4293004354', 'cluster_addr': '172.21.15.63:6811/4293004354', 'heartbeat_back_addr': '172.21.15.63:6815/4293004354', 'heartbeat_front_addr': '172.21.15.63:6813/4293004354', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '1cec9702-fa09-4d7c-b111-7d10b1950cfe', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1789187713}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1789187713}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1789187713}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1789187713}]}, 'public_addr': '172.21.15.99:6801/1789187713', 'cluster_addr': '172.21.15.99:6803/1789187713', 'heartbeat_back_addr': '172.21.15.99:6807/1789187713', 'heartbeat_front_addr': '172.21.15.99:6805/1789187713', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7c0b908a-65c9-426c-867c-cc8ed65084b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6809', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3410135021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6813', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3410135021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6821', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3410135021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6817', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3410135021}]}, 'public_addr': '172.21.15.99:6811/3410135021', 'cluster_addr': '172.21.15.99:6815/3410135021', 'heartbeat_back_addr': '172.21.15.99:6823/3410135021', 'heartbeat_front_addr': '172.21.15.99:6819/3410135021', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78d5990a-4535-4110-ba04-88f01ab9fe28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6810', 'nonce': 4152410871}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6814', 'nonce': 4152410871}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6822', 'nonce': 4152410871}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6818', 'nonce': 4152410871}]}, 'public_addr': '172.21.15.99:6810/4152410871', 'cluster_addr': '172.21.15.99:6814/4152410871', 'heartbeat_back_addr': '172.21.15.99:6822/4152410871', 'heartbeat_front_addr': '172.21.15.99:6818/4152410871', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '53f8511c-729f-4423-8eac-813c01f8f72b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2923799929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2923799929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2923799929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2923799929}]}, 'public_addr': '172.21.15.99:6825/2923799929', 'cluster_addr': '172.21.15.99:6827/2923799929', 'heartbeat_back_addr': '172.21.15.99:6831/2923799929', 'heartbeat_front_addr': '172.21.15.99:6829/2923799929', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.267895+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.161313+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.447964+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.021320+0000', 'dead_epoch': 48}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.883961+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.938146+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.627237+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.887523+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:27:25.369 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:27:25.370 INFO:teuthology.orchestra.run.smithi063.stdout:3975 write 0x9fce0d0 thru 0x9fd39a6 (0x58d7 bytes) 2024-09-13T09:27:25.370 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.370 INFO:teuthology.orchestra.run.smithi063.stdout:3978 read 0x41e60ac thru 0x41e6632 (0x587 bytes) 2024-09-13T09:27:25.370 INFO:teuthology.orchestra.run.smithi063.stdout:3979 read 0x8660d94 thru 0x866f24c (0xe4b9 bytes) 2024-09-13T09:27:25.370 INFO:teuthology.orchestra.run.smithi063.stdout:3980 read 0x45a7ad7 thru 0x45b0069 (0x8593 bytes) 2024-09-13T09:27:25.371 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 20.58287292, 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.372 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:25.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:25.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124576 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.186367362999999, 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050917829999999999, 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124576 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134158+0000", 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134164+0000", 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.083e-06 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134180+0000", 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5177000000000001e-05 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134192+0000", 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.216e-05 2024-09-13T09:27:25.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138437+0000", 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042449430000000002 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138535+0000", 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7962999999999997e-05 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139251+0000", 2024-09-13T09:27:25.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071628499999999999 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124584 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.185196803, 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004962073, 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.383 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124584 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135327+0000", 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135341+0000", 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3454e-05 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135366+0000", 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.51e-05 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135378+0000", 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2048000000000001e-05 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139290+0000", 2024-09-13T09:27:25.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039123580000000003 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139394+0000", 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010412999999999999 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140292+0000", 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00089737400000000004 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.387 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124586 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.18513694, 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062408630000000001, 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124586 2024-09-13T09:27:25.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135389+0000", 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135396+0000", 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7309999999999996e-06 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135404+0000", 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5369999999999997e-06 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135411+0000", 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140323+0000", 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049127659999999998 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140390+0000", 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6631999999999995e-05 2024-09-13T09:27:25.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141630+0000", 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012403480000000001 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124592 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:25.392 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.185063112000002, 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0076439699999999999, 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124592 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135469+0000", 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135500+0000", 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1149000000000003e-05 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135513+0000", 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2771e-05 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142205+0000", 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066924239999999998 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142329+0000", 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012366100000000001 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143107+0000", 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077859999999999995 2024-09-13T09:27:25.396 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124596 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.184951302000002, 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078700200000000001, 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.397 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124596 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135574+0000", 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135581+0000", 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3280000000000002e-06 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135597+0000", 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5996999999999999e-05 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135608+0000", 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.166e-05 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143162+0000", 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0075534349999999998 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143306+0000", 2024-09-13T09:27:25.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014448900000000001 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143445+0000", 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013898200000000001 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124608 3.7 3:e4e5f482:::rbd_data.1118f3eff690.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:25.401 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.175473386, 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062754919999999997, 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124608 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:25.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145052+0000", 2024-09-13T09:27:25.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145531+0000", 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00047855199999999998 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145542+0000", 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0961000000000001e-05 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145551+0000", 2024-09-13T09:27:25.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:27:25.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145661+0000", 2024-09-13T09:27:25.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109726 2024-09-13T09:27:25.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145992+0000", 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033164000000000002 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.146845+0000", 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085247199999999995 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.148693+0000", 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018483460000000001 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151235+0000", 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002542159 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151297+0000", 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1710000000000004e-05 2024-09-13T09:27:25.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151329+0000", 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1730000000000003e-05 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124700 3.1e 3:78e395d8:::rbd_data.1118f3eff690.0000000000000080:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.018230191000001, 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011658599, 2024-09-13T09:27:25.412 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124700 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302294+0000", 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302307+0000", 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3602e-05 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302336+0000", 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8116999999999999e-05 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302348+0000", 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2792000000000001e-05 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302394+0000", 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5797000000000002e-05 2024-09-13T09:27:25.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302505+0000", 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111286 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.304334+0000", 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828325 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313641+0000", 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093073449999999999 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313729+0000", 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8305000000000002e-05 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313834+0000", 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010443899999999999 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313955+0000", 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121142 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:124700 3.1e e78/77 ondisk, result = 0)", 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.012345452999998, 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057895580000000002, 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:25.418 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308180+0000", 2024-09-13T09:27:25.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308188+0000", 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308203+0000", 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4977000000000001e-05 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308216+0000", 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2855e-05 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313698+0000", 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054822960000000002 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313716+0000", 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8167000000000001e-05 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313971+0000", 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025462500000000002 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125238 3.18 3:1cc4019a:::rbd_data.1118bda513e1.0000000000000005:head [stat,write 16509626~3298] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.279206694999999, 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065355600000000002, 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.422 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125238 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041315+0000", 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999957 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041336+0000", 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1073e-05 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041373+0000", 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7039000000000002e-05 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041394+0000", 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0477000000000001e-05 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041475+0000", 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0952999999999996e-05 2024-09-13T09:27:25.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041481+0000", 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045492+0000", 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040115339999999998 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045585+0000", 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2255000000000004e-05 2024-09-13T09:27:25.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045728+0000", 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014281499999999999 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047526+0000", 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017983400000000001 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047681+0000", 2024-09-13T09:27:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015514200000000001 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047777+0000", 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6415000000000004e-05 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047855+0000", 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7985999999999995e-05 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126514 3.1e 3:7f6180ef:::rbd_object_map.1118413dc0c1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0935287680000005, 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0077678299999999999, 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126514 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226997+0000", 2024-09-13T09:27:25.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227004+0000", 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.187e-06 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227027+0000", 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3254000000000001e-05 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227034+0000", 2024-09-13T09:27:25.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6479999999999997e-06 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227078+0000", 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227182+0000", 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104095 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227687+0000", 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00050565500000000004 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.233076+0000", 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053885319999999997 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234572+0000", 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014962490000000001 2024-09-13T09:27:25.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234697+0000", 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000124626 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234766+0000", 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8842000000000003e-05 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.434 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126516 3.1e 3:7f6180ef:::rbd_object_map.1118413dc0c1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0931517890000002, 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078524679999999996, 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126516 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227373+0000", 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227386+0000", 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2916e-05 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227409+0000", 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3268000000000002e-05 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227425+0000", 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5798e-05 2024-09-13T09:27:25.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.230833+0000", 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034084919999999999 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.231022+0000", 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000188763 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.231925+0000", 2024-09-13T09:27:25.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00090269899999999997 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234795+0000", 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028700589999999999 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235100+0000", 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030537400000000002 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235153+0000", 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2222999999999998e-05 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235227+0000", 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4719000000000001e-05 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:127527 3.8 3:149f3857:::rbd_header.11181a204f0d:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:19.878053+0000", 2024-09-13T09:27:25.440 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4424738130000003, 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050577039999999997, 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 127527 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878053+0000", 2024-09-13T09:27:25.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878053+0000", 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878049+0000", 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878067+0000", 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8119999999999999e-05 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878100+0000", 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3108000000000002e-05 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878117+0000", 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6197000000000001e-05 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878195+0000", 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8423000000000005e-05 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878201+0000", 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8409999999999996e-06 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.880505+0000", 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023038310000000001 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.880649+0000", 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014465100000000001 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.881167+0000", 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00051791999999999995 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.882310+0000", 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001142426 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.883018+0000", 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00070869200000000002 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.883072+0000", 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3570999999999999e-05 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.883110+0000", 2024-09-13T09:27:25.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8491999999999997e-05 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128783 3.7 3:e5fee451:::rbd_data.11181500dc3e.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.598020+0000", 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.722506804, 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043823050000000004, 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128783 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598020+0000", 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598020+0000", 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598018+0000", 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598361+0000", 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00034352599999999998 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598380+0000", 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8675999999999999e-05 2024-09-13T09:27:25.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598388+0000", 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3389999999999993e-06 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598454+0000", 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6235999999999994e-05 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598629+0000", 2024-09-13T09:27:25.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017499400000000001 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.598851+0000", 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000221241 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.600519+0000", 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001668016 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.602324+0000", 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001804932 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.602368+0000", 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4669999999999998e-05 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.602402+0000", 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3736999999999999e-05 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.452 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128801 3.7 3:e5fee451:::rbd_data.11181500dc3e.0000000000000008:head [stat,write 32768~14443] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.622455+0000", 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.69807142499999997, 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044694649999999997, 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128801 2024-09-13T09:27:25.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622455+0000", 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622455+0000", 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622453+0000", 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622473+0000", 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9507000000000001e-05 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622494+0000", 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1455999999999999e-05 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622504+0000", 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5510000000000005e-06 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622569+0000", 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5289000000000005e-05 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622653+0000", 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3283999999999997e-05 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622883+0000", 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023047700000000001 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.624457+0000", 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015737609999999999 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626780+0000", 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023235909999999999 2024-09-13T09:27:25.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626842+0000", 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1818999999999999e-05 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626925+0000", 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2368999999999997e-05 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.458 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128961 3.b 3:d06f0007:::rbd_data.11181500dc3e.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.473259971, 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010200722000000001, 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128961 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847265+0000", 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847762+0000", 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00049738799999999998 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847787+0000", 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4451999999999999e-05 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847799+0000", 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2140999999999999e-05 2024-09-13T09:27:25.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847886+0000", 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7020000000000004e-05 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.848076+0000", 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019032700000000001 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852181+0000", 2024-09-13T09:27:25.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041049830000000004 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857348+0000", 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051670809999999996 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857367+0000", 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8533e-05 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857428+0000", 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1334999999999995e-05 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857467+0000", 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9286000000000003e-05 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128963 3.1e 3:7a2537bd:::rbd_data.11181500dc3e.000000000000005b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888542+0000", 2024-09-13T09:27:25.464 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.43198489299999998, 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049817630000000002, 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128963 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888542+0000", 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888542+0000", 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888540+0000", 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888553+0000", 2024-09-13T09:27:25.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3329999999999999e-05 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888586+0000", 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2925000000000001e-05 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888603+0000", 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6730000000000001e-05 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888762+0000", 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015963800000000001 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888978+0000", 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000216176 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889216+0000", 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023740299999999999 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.891435+0000", 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002219412 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893451+0000", 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020155199999999998 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893488+0000", 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7635999999999997e-05 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893523+0000", 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5064000000000002e-05 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129048 3.b 3:d1c06694:::rbd_data.11181500dc3e.0000000000000088:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.27822347800000002, 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014253025000000001, 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.470 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129048 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042302+0000", 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042309+0000", 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.601e-06 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042322+0000", 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3529999999999999e-05 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042327+0000", 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042399+0000", 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1208e-05 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042524+0000", 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012532799999999999 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.044815+0000", 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022907729999999999 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056394+0000", 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011579271 2024-09-13T09:27:25.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056445+0000", 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0994e-05 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056500+0000", 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5442999999999998e-05 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056556+0000", 2024-09-13T09:27:25.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5702000000000003e-05 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129113 3.7 3:e0643571:::rbd_data.11181500dc3e.000000000000016e:head [set-alloc-hint object_size 524288 write_size 524288,write 98304~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.18717536200000001, 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057129440000000002, 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129113 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133348+0000", 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133380+0000", 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2150000000000002e-05 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133409+0000", 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8924e-05 2024-09-13T09:27:25.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133423+0000", 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3773e-05 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133527+0000", 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010395600000000001 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133780+0000", 2024-09-13T09:27:25.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025285500000000002 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134012+0000", 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023174799999999999 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.137751+0000", 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037395369999999998 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.138980+0000", 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001228282 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.139035+0000", 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5652999999999997e-05 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.139064+0000", 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8969e-05 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.481 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129140 3.7 3:e0643571:::rbd_data.11181500dc3e.000000000000016e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.10922683599999999, 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086215130000000008, 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129140 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.482 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211298+0000", 2024-09-13T09:27:25.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211307+0000", 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6829999999999997e-06 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212397+0000", 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010900009999999999 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212409+0000", 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2746e-05 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212490+0000", 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0668999999999996e-05 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.213419+0000", 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092909700000000004 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.213568+0000", 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000148531 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.214326+0000", 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00075828500000000004 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219843+0000", 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055167430000000002 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219875+0000", 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1869000000000003e-05 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219921+0000", 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6560000000000001e-05 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.487 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:25.488 INFO:teuthology.orchestra.run.smithi063.stdout:3981 trunc from 0x9fd39a7 to 0x50de52b 2024-09-13T09:27:25.489 INFO:teuthology.orchestra.run.smithi063.stdout:3982 trunc from 0x50de52b to 0xdc5ba63 2024-09-13T09:27:25.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:25.490 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:25.359+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:25.491 INFO:teuthology.orchestra.run.smithi063.stdout:3983 write 0x3589eb7 thru 0x358d201 (0x334b bytes) 2024-09-13T09:27:25.491 INFO:teuthology.orchestra.run.smithi063.stdout:3984 write 0x35f928c thru 0x360904a (0xfdbf bytes) 2024-09-13T09:27:25.491 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3987 read 0x177580f thru 0x178060c (0xadfe bytes) 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3989 trunc from 0xdc5ba63 to 0xde6c4dc 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3990 trunc from 0xde6c4dc to 0xed2011a 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3992 punch from 0x1d39c38 to 0x1d48e8e, (0xf256 bytes) 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3993 read 0x7905d0e thru 0x7909610 (0x3903 bytes) 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3994 trunc from 0xed2011a to 0xb18fa0c 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3995 write 0xd41b27f thru 0xd422406 (0x7188 bytes) 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3996 write 0x8e7062b thru 0x8e76f4a (0x6920 bytes) 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.492 INFO:teuthology.orchestra.run.smithi063.stdout:3998 trunc from 0xd422407 to 0x2b14ca9 2024-09-13T09:27:25.503 INFO:teuthology.orchestra.run.smithi063.stdout:3999 punch from 0x27ec298 to 0x27f5f9f, (0x9d07 bytes) 2024-09-13T09:27:25.504 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:25.495+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:25.525 INFO:teuthology.orchestra.run.smithi063.stdout:4000 write 0x8affd9f thru 0x8b085ce (0x8830 bytes) 2024-09-13T09:27:25.535 INFO:teuthology.orchestra.run.smithi063.stdout:4001 writesame 0x75ef8ca thru 0x75f5cbd data_size 0x2(0x63f4 bytes) 2024-09-13T09:27:25.549 INFO:teuthology.orchestra.run.smithi063.stdout:4002 write 0x4fab857 thru 0x4fb2b02 (0x72ac bytes) 2024-09-13T09:27:25.572 INFO:teuthology.orchestra.run.smithi063.stdout:4003 writesame 0xa4e5e8c thru 0xa4e9127 data_size 0x52(0x329c bytes) 2024-09-13T09:27:25.572 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.572 INFO:teuthology.orchestra.run.smithi063.stdout:4010 read 0xa3c30bf thru 0xa3d0566 (0xd4a8 bytes) 2024-09-13T09:27:25.573 INFO:teuthology.orchestra.run.smithi063.stdout:4011 writesame 0x559f76e thru 0x55a08ed data_size 0x40(0x1180 bytes) 2024-09-13T09:27:25.583 INFO:teuthology.orchestra.run.smithi063.stdout:4012 write 0xa4bfe11 thru 0xa4c0e29 (0x1019 bytes) 2024-09-13T09:27:25.585 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:25.590 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:25.587+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:25.605 INFO:teuthology.orchestra.run.smithi063.stdout:4014 writesame 0xad21ed9 thru 0xad2ccbf data_size 0x1(0xade7 bytes) 2024-09-13T09:27:25.605 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.605 INFO:teuthology.orchestra.run.smithi063.stdout:4018 punch from 0x378e707 to 0x37936d2, (0x4fcb bytes) 2024-09-13T09:27:25.620 INFO:teuthology.orchestra.run.smithi063.stdout:4020 trunc from 0xad2ccc0 to 0x1242d80 2024-09-13T09:27:25.632 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:25.632 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:25.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129345 3.1f 3:fc9e9d36:::rbd_data.11181500dc3e.0000000000000038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.621946+0000", 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0017875300000000001, 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018140890000000001, 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129345 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621946+0000", 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621946+0000", 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621945+0000", 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:25.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621952+0000", 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.097e-06 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621964+0000", 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2607e-05 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621971+0000", 2024-09-13T09:27:25.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8090000000000004e-06 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622011+0000", 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0185000000000002e-05 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622106+0000", 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4344000000000003e-05 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622220+0000", 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0001142 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.623640+0000", 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001419963 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129349 3.1b 3:df15d050:::rbd_data.11181500dc3e.00000000000000a4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.637 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.621985+0000", 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0017485479999999999, 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001860593, 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129349 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621985+0000", 2024-09-13T09:27:25.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621985+0000", 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621985+0000", 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621988+0000", 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0910000000000001e-06 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621994+0000", 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0299999999999999e-06 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.621999+0000", 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6239999999999999e-06 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622045+0000", 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5651e-05 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622130+0000", 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4767999999999999e-05 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622220+0000", 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0461e-05 2024-09-13T09:27:25.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.623623+0000", 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014026589999999999 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129353 3.1 3:875c0977:::rbd_data.11181500dc3e.00000000000000ea:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.622015+0000", 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0017188629999999999, 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001908711, 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:25.642 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129353 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622015+0000", 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:25.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622015+0000", 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622015+0000", 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622017+0000", 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622024+0000", 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3099999999999997e-06 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622029+0000", 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0050000000000004e-06 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622083+0000", 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4035e-05 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622159+0000", 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5858000000000005e-05 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.622239+0000", 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9994999999999996e-05 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.623773+0000", 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001534615 2024-09-13T09:27:25.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.647 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:25.647 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.647 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:25.647 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:25.647 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 3 2024-09-13T09:27:25.647 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:25.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:25.649 INFO:teuthology.orchestra.run.smithi063.stdout:4021 read 0x119270d thru 0x119dcb5 (0xb5a9 bytes) 2024-09-13T09:27:25.649 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.649 INFO:teuthology.orchestra.run.smithi063.stdout:4023 read 0x1107d76 thru 0x111354f (0xb7da bytes) 2024-09-13T09:27:25.649 INFO:teuthology.orchestra.run.smithi063.stdout:4025 writesame 0x781e4c thru 0x78952d data_size 0x1(0x76e2 bytes) 2024-09-13T09:27:25.688 INFO:teuthology.orchestra.run.smithi063.stdout:4026 write 0x91bed69 thru 0x91ce1de (0xf476 bytes) 2024-09-13T09:27:25.691 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:25.687+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:25.704 INFO:teuthology.orchestra.run.smithi063.stdout:4027 trunc from 0x91ce1df to 0xbb8c815 2024-09-13T09:27:25.733 INFO:teuthology.orchestra.run.smithi063.stdout:4028 trunc from 0xbb8c815 to 0x1642dbf 2024-09-13T09:27:25.746 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:25.774 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:25.783 INFO:teuthology.orchestra.run.smithi063.stdout:4029 write 0x92b886b thru 0x92c4e18 (0xc5ae bytes) 2024-09-13T09:27:25.789 INFO:teuthology.orchestra.run.smithi063.stdout:4030 writesame 0x78bc106 thru 0x78cbfb0 data_size 0x55(0xfeab bytes) 2024-09-13T09:27:25.792 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:25.787+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:25.797 INFO:teuthology.orchestra.run.smithi063.stdout:4031 punch from 0xee6242 to 0xef3229, (0xcfe7 bytes) 2024-09-13T09:27:25.798 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:25.798 INFO:teuthology.orchestra.run.smithi063.stdout:4032 write 0x4fd73d1 thru 0x4fdbb1e (0x474e bytes) 2024-09-13T09:27:25.812 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:25.812 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:25.813 INFO:teuthology.orchestra.run.smithi063.stdout:4033 trunc from 0x92c4e19 to 0x348aff3 2024-09-13T09:27:25.819 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:25.827 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:25.831 INFO:teuthology.orchestra.run.smithi063.stdout:4036 writesame 0x30ac10f thru 0x30afbf9 data_size 0x1(0x3aeb bytes) 2024-09-13T09:27:25.847 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:25.864 INFO:teuthology.orchestra.run.smithi063.stdout:4037 trunc from 0x348aff3 to 0xe8a223a 2024-09-13T09:27:25.870 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:25.875 INFO:teuthology.orchestra.run.smithi063.stdout:4038 read 0xa5d22f thru 0xa68bd3 (0xb9a5 bytes) 2024-09-13T09:27:25.876 INFO:teuthology.orchestra.run.smithi063.stdout:4039 write 0x8957b49 thru 0x895a245 (0x26fd bytes) 2024-09-13T09:27:25.884 INFO:teuthology.orchestra.run.smithi063.stdout:4041 punch from 0x66e9ced to 0x66f8e1e, (0xf131 bytes) 2024-09-13T09:27:25.889 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:25.892 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:27:25.892 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:25.887+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:25.896 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:25.899 INFO:teuthology.orchestra.run.smithi063.stdout:4042 trunc from 0xe8a223a to 0x539df5f 2024-09-13T09:27:25.909 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-13T09:27:24.544718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1344053949,"num_objects":1192,"num_object_clones":255,"num_object_copies":2384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":1192,"num_whiteouts":216,"num_read":163773,"num_read_kb":520352,"num_write":14764,"num_write_kb":320868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1319,"num_bytes_recovered":100528566,"num_keys_recovered":340,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11182,"ondisk_log_size":11182,"up":82,"acting":69,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":316152,"kb_used_data":49792,"kb_used_omap":23,"kb_used_meta":266216,"kb_avail":660286728,"statfs":{"total":676457349120,"available":676133609472,"internally_reserved":0,"allocated":50987008,"data_stored":300372044,"data_compressed":22983167,"data_compressed_allocated":27590656,"data_compressed_original":284237205,"omap_allocated":24352,"internal_metadata":272605408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":66654795,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":2335,"num_read_kb":7022,"num_write":463,"num_write_kb":5901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":406,"num_bytes_recovered":32806108,"num_keys_recovered":135,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001206"},"pg_stats":[{"pgid":"3.1f","version":"92'218","reported_seq":896,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.559670+0000","last_change":"2024-09-13T09:26:39.600998+0000","last_active":"2024-09-13T09:27:21.559670+0000","last_peered":"2024-09-13T09:27:21.559670+0000","last_clean":"2024-09-13T09:27:21.559670+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:27:21.559670+0000","last_undegraded":"2024-09-13T09:27:21.559670+0000","last_fullsized":"2024-09-13T09:27:21.559670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":38105270,"num_objects":39,"num_object_clones":12,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":9,"num_read":2370,"num_read_kb":13284,"num_write":239,"num_write_kb":6276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"89'468","reported_seq":4764,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.882078+0000","last_change":"2024-09-13T09:27:05.744365+0000","last_active":"2024-09-13T09:27:19.882078+0000","last_peered":"2024-09-13T09:27:19.882078+0000","last_clean":"2024-09-13T09:27:19.882078+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:27:19.882078+0000","last_undegraded":"2024-09-13T09:27:19.882078+0000","last_fullsized":"2024-09-13T09:27:19.882078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":27252671,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":2,"num_read":5798,"num_read_kb":11596,"num_write":743,"num_write_kb":4707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"92'297","reported_seq":2602,"reported_epoch":92,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:27:21.227384+0000","last_change":"2024-09-13T09:27:21.188550+0000","last_active":"2024-09-13T09:27:21.227384+0000","last_peered":"2024-09-13T09:27:21.227384+0000","last_clean":"2024-09-13T09:27:18.123326+0000","last_became_active":"2024-09-13T09:27:21.187730+0000","last_became_peered":"2024-09-13T09:27:21.187730+0000","last_unstale":"2024-09-13T09:27:21.227384+0000","last_undegraded":"2024-09-13T09:27:21.170107+0000","last_fullsized":"2024-09-13T09:27:21.169044+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:29.630433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":54551837,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":83,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":8,"num_read":3172,"num_read_kb":15523,"num_write":380,"num_write_kb":9013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5],"avail_no_missing":["5","4"],"object_location_counts":[{"shards":"4,5","objects":38},{"shards":"5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"92'359","reported_seq":3897,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-09-13T09:27:21.459811+0000","last_change":"2024-09-13T09:27:21.419538+0000","last_active":"2024-09-13T09:27:21.459811+0000","last_peered":"2024-09-13T09:27:21.459811+0000","last_clean":"2024-09-13T09:27:18.805991+0000","last_became_active":"2024-09-13T09:27:21.183704+0000","last_became_peered":"2024-09-13T09:27:21.183704+0000","last_unstale":"2024-09-13T09:27:21.459811+0000","last_undegraded":"2024-09-13T09:27:21.459811+0000","last_fullsized":"2024-09-13T09:27:21.170314+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:39.733145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":56460366,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":6,"num_read":4098,"num_read_kb":12858,"num_write":461,"num_write_kb":9378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6976317,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5],"avail_no_missing":["5","1","4"],"object_location_counts":[{"shards":"5","objects":34}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"85'285","reported_seq":1551,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-09-13T09:27:21.394132+0000","last_change":"2024-09-13T09:27:21.394406+0000","last_active":"2024-09-13T09:27:21.394132+0000","last_peered":"2024-09-13T09:27:21.394132+0000","last_clean":"2024-09-13T09:27:17.860819+0000","last_became_active":"2024-09-13T09:27:21.183759+0000","last_became_peered":"2024-09-13T09:27:21.183759+0000","last_unstale":"2024-09-13T09:27:21.394132+0000","last_undegraded":"2024-09-13T09:27:21.394132+0000","last_fullsized":"2024-09-13T09:27:21.164030+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:52.329752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9317689,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2180,"num_read_kb":8843,"num_write":345,"num_write_kb":5217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":6078834,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0","objects":29}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"89'328","reported_seq":10241,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-09-13T09:27:21.584180+0000","last_change":"2024-09-13T09:27:21.406516+0000","last_active":"2024-09-13T09:27:21.584180+0000","last_peered":"2024-09-13T09:27:21.584180+0000","last_clean":"2024-09-13T09:27:18.753710+0000","last_became_active":"2024-09-13T09:27:21.173692+0000","last_became_peered":"2024-09-13T09:27:21.173692+0000","last_unstale":"2024-09-13T09:27:21.584180+0000","last_undegraded":"2024-09-13T09:27:21.584180+0000","last_fullsized":"2024-09-13T09:27:21.157921+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:11:22.388151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":44917002,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":6,"num_read":11445,"num_read_kb":19191,"num_write":346,"num_write_kb":5397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":3840562,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1","objects":39}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"87'337","reported_seq":1572,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.464047+0000","last_change":"2024-09-13T09:27:05.741503+0000","last_active":"2024-09-13T09:27:21.464047+0000","last_peered":"2024-09-13T09:27:21.464047+0000","last_clean":"2024-09-13T09:27:21.464047+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:27:21.464047+0000","last_undegraded":"2024-09-13T09:27:21.464047+0000","last_fullsized":"2024-09-13T09:27:21.464047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":25108629,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":6,"num_read":2476,"num_read_kb":7958,"num_write":344,"num_write_kb":6942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"89'312","reported_seq":1235,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.880411+0000","last_change":"2024-09-13T09:27:03.781188+0000","last_active":"2024-09-13T09:27:19.880411+0000","last_peered":"2024-09-13T09:27:19.880411+0000","last_clean":"2024-09-13T09:27:19.880411+0000","last_became_active":"2024-09-13T09:27:03.711948+0000","last_became_peered":"2024-09-13T09:27:03.711948+0000","last_unstale":"2024-09-13T09:27:19.880411+0000","last_undegraded":"2024-09-13T09:27:19.880411+0000","last_fullsized":"2024-09-13T09:27:19.880411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:42.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":24810743,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":2846,"num_read_kb":11789,"num_write":330,"num_write_kb":10779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":8502479,"num_keys_recovered":39,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"92'480","reported_seq":1214,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.549269+0000","last_change":"2024-09-13T09:26:42.575210+0000","last_active":"2024-09-13T09:27:21.549269+0000","last_peered":"2024-09-13T09:27:21.549269+0000","last_clean":"2024-09-13T09:27:21.549269+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:27:21.549269+0000","last_undegraded":"2024-09-13T09:27:21.549269+0000","last_fullsized":"2024-09-13T09:27:21.549269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:47.404587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8448779,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":4,"num_read":2665,"num_read_kb":4443,"num_write":587,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"92'236","reported_seq":4739,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.168775+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:27:21.168775+0000","last_peered":"2024-09-13T09:27:21.168775+0000","last_clean":"2024-09-13T09:27:21.168775+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:27:21.168775+0000","last_undegraded":"2024-09-13T09:27:21.168775+0000","last_fullsized":"2024-09-13T09:27:21.168775+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":48906580,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":6,"num_read":4791,"num_read_kb":23892,"num_write":325,"num_write_kb":11536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'208","reported_seq":6650,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156866+0000","last_change":"2024-09-13T09:27:05.750137+0000","last_active":"2024-09-13T09:27:20.156866+0000","last_peered":"2024-09-13T09:27:20.156866+0000","last_clean":"2024-09-13T09:27:20.156866+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:27:20.156866+0000","last_undegraded":"2024-09-13T09:27:20.156866+0000","last_fullsized":"2024-09-13T09:27:20.156866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":23304536,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":5,"num_read":6764,"num_read_kb":20582,"num_write":235,"num_write_kb":5907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"85'186","reported_seq":3617,"reported_epoch":92,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:27:21.184467+0000","last_change":"2024-09-13T09:27:21.184467+0000","last_active":"2024-09-13T09:27:21.184467+0000","last_peered":"2024-09-13T09:27:21.184467+0000","last_clean":"2024-09-13T09:27:13.830682+0000","last_became_active":"2024-09-13T09:27:21.183775+0000","last_became_peered":"2024-09-13T09:27:21.183775+0000","last_unstale":"2024-09-13T09:27:21.184467+0000","last_undegraded":"2024-09-13T09:27:21.161615+0000","last_fullsized":"2024-09-13T09:27:21.158993+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:51.187021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":43978384,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":78,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":3413,"num_read_kb":10230,"num_write":201,"num_write_kb":4175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":["7","4"],"object_location_counts":[{"shards":"4,7","objects":35}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":101,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156708+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:27:20.156708+0000","last_peered":"2024-09-13T09:27:20.156708+0000","last_clean":"2024-09-13T09:27:20.156708+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:27:20.156708+0000","last_undegraded":"2024-09-13T09:27:20.156708+0000","last_fullsized":"2024-09-13T09:27:20.156708+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"89'274","reported_seq":1042,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.169036+0000","last_change":"2024-09-13T09:27:05.753951+0000","last_active":"2024-09-13T09:27:21.169036+0000","last_peered":"2024-09-13T09:27:21.169036+0000","last_clean":"2024-09-13T09:27:21.169036+0000","last_became_active":"2024-09-13T09:27:05.753131+0000","last_became_peered":"2024-09-13T09:27:05.753131+0000","last_unstale":"2024-09-13T09:27:21.169036+0000","last_undegraded":"2024-09-13T09:27:21.169036+0000","last_fullsized":"2024-09-13T09:27:21.169036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:21.776084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":66881518,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":7,"num_read":951,"num_read_kb":5336,"num_write":337,"num_write_kb":6753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156592+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:27:20.156592+0000","last_peered":"2024-09-13T09:27:20.156592+0000","last_clean":"2024-09-13T09:27:20.156592+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:27:20.156592+0000","last_undegraded":"2024-09-13T09:27:20.156592+0000","last_fullsized":"2024-09-13T09:27:20.156592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"87'291","reported_seq":2358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.157629+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:27:20.157629+0000","last_peered":"2024-09-13T09:27:20.157629+0000","last_clean":"2024-09-13T09:27:20.157629+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:27:20.157629+0000","last_undegraded":"2024-09-13T09:27:20.157629+0000","last_fullsized":"2024-09-13T09:27:20.157629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":23702068,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":3,"num_read":3600,"num_read_kb":13521,"num_write":370,"num_write_kb":18384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.159306+0000","last_change":"2024-09-13T09:27:20.158360+0000","last_active":"2024-09-13T09:27:20.159306+0000","last_peered":"2024-09-13T09:27:20.159306+0000","last_clean":"2024-09-13T09:27:20.159306+0000","last_became_active":"2024-09-13T09:27:20.157982+0000","last_became_peered":"2024-09-13T09:27:20.157982+0000","last_unstale":"2024-09-13T09:27:20.159306+0000","last_undegraded":"2024-09-13T09:27:20.159306+0000","last_fullsized":"2024-09-13T09:27:20.159306+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:38:59.030047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'249","reported_seq":3772,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.881156+0000","last_change":"2024-09-13T09:27:05.741382+0000","last_active":"2024-09-13T09:27:19.881156+0000","last_peered":"2024-09-13T09:27:19.881156+0000","last_clean":"2024-09-13T09:27:19.881156+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:27:19.881156+0000","last_undegraded":"2024-09-13T09:27:19.881156+0000","last_fullsized":"2024-09-13T09:27:19.881156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":68403673,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":9,"num_read":4332,"num_read_kb":19799,"num_write":248,"num_write_kb":6679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.880770+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:27:19.880770+0000","last_peered":"2024-09-13T09:27:19.880770+0000","last_clean":"2024-09-13T09:27:19.880770+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:27:19.880770+0000","last_undegraded":"2024-09-13T09:27:19.880770+0000","last_fullsized":"2024-09-13T09:27:19.880770+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"92'308","reported_seq":3569,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.474661+0000","last_change":"2024-09-13T09:26:42.570158+0000","last_active":"2024-09-13T09:27:21.474661+0000","last_peered":"2024-09-13T09:27:21.474661+0000","last_clean":"2024-09-13T09:27:21.474661+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:27:21.474661+0000","last_undegraded":"2024-09-13T09:27:21.474661+0000","last_fullsized":"2024-09-13T09:27:21.474661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41870686,"num_objects":47,"num_object_clones":10,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":4855,"num_read_kb":16906,"num_write":303,"num_write_kb":4449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"92'577","reported_seq":1585,"reported_epoch":92,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:27:21.562968+0000","last_change":"2024-09-13T09:27:21.215223+0000","last_active":"2024-09-13T09:27:21.562968+0000","last_peered":"2024-09-13T09:27:21.562968+0000","last_clean":"2024-09-13T09:27:17.860651+0000","last_became_active":"2024-09-13T09:27:21.214106+0000","last_became_peered":"2024-09-13T09:27:21.214106+0000","last_unstale":"2024-09-13T09:27:21.562968+0000","last_undegraded":"2024-09-13T09:27:21.165995+0000","last_fullsized":"2024-09-13T09:27:21.163630+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:47.150317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829875,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":60,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":4113,"num_read_kb":12664,"num_write":903,"num_write_kb":5032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":7377,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,4","objects":22}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":232,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156483+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:27:20.156483+0000","last_peered":"2024-09-13T09:27:20.156483+0000","last_clean":"2024-09-13T09:27:20.156483+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:27:20.156483+0000","last_undegraded":"2024-09-13T09:27:20.156483+0000","last_fullsized":"2024-09-13T09:27:20.156483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.157672+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:27:20.157672+0000","last_peered":"2024-09-13T09:27:20.157672+0000","last_clean":"2024-09-13T09:27:20.157672+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:27:20.157672+0000","last_undegraded":"2024-09-13T09:27:20.157672+0000","last_fullsized":"2024-09-13T09:27:20.157672+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"89'696","reported_seq":7482,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.157501+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:27:20.157501+0000","last_peered":"2024-09-13T09:27:20.157501+0000","last_clean":"2024-09-13T09:27:20.157501+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:27:20.157501+0000","last_undegraded":"2024-09-13T09:27:20.157501+0000","last_fullsized":"2024-09-13T09:27:20.157501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":37872699,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":9955,"num_read_kb":15166,"num_write":1109,"num_write_kb":14153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156735+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:27:20.156735+0000","last_peered":"2024-09-13T09:27:20.156735+0000","last_clean":"2024-09-13T09:27:20.156735+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:27:20.156735+0000","last_undegraded":"2024-09-13T09:27:20.156735+0000","last_fullsized":"2024-09-13T09:27:20.156735+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'326","reported_seq":9824,"reported_epoch":92,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-13T09:27:21.577836+0000","last_change":"2024-09-13T09:27:21.399103+0000","last_active":"2024-09-13T09:27:21.577836+0000","last_peered":"2024-09-13T09:27:21.577836+0000","last_clean":"2024-09-13T09:27:17.854377+0000","last_became_active":"2024-09-13T09:27:21.187402+0000","last_became_peered":"2024-09-13T09:27:21.187402+0000","last_unstale":"2024-09-13T09:27:21.577836+0000","last_undegraded":"2024-09-13T09:27:21.577836+0000","last_fullsized":"2024-09-13T09:27:21.163241+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:20.254725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":83356232,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":9701,"num_read_kb":37472,"num_write":429,"num_write_kb":28738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":21133266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0","objects":28},{"shards":"0,4","objects":17}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.158176+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:27:20.158176+0000","last_peered":"2024-09-13T09:27:20.158176+0000","last_clean":"2024-09-13T09:27:20.158176+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:27:20.158176+0000","last_undegraded":"2024-09-13T09:27:20.158176+0000","last_fullsized":"2024-09-13T09:27:20.158176+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'187","reported_seq":4148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.880508+0000","last_change":"2024-09-13T09:27:05.745925+0000","last_active":"2024-09-13T09:27:19.880508+0000","last_peered":"2024-09-13T09:27:19.880508+0000","last_clean":"2024-09-13T09:27:19.880508+0000","last_became_active":"2024-09-13T09:27:05.744803+0000","last_became_peered":"2024-09-13T09:27:05.744803+0000","last_unstale":"2024-09-13T09:27:19.880508+0000","last_undegraded":"2024-09-13T09:27:19.880508+0000","last_fullsized":"2024-09-13T09:27:19.880508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:40.185419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":23509317,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":3933,"num_read_kb":4516,"num_write":195,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156566+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:27:20.156566+0000","last_peered":"2024-09-13T09:27:20.156566+0000","last_clean":"2024-09-13T09:27:20.156566+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:27:20.156566+0000","last_undegraded":"2024-09-13T09:27:20.156566+0000","last_fullsized":"2024-09-13T09:27:20.156566+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"90'320","reported_seq":1404,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.883074+0000","last_change":"2024-09-13T09:27:05.745269+0000","last_active":"2024-09-13T09:27:19.883074+0000","last_peered":"2024-09-13T09:27:19.883074+0000","last_clean":"2024-09-13T09:27:19.883074+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:27:19.883074+0000","last_undegraded":"2024-09-13T09:27:19.883074+0000","last_fullsized":"2024-09-13T09:27:19.883074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":21357924,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":6,"num_read":2519,"num_read_kb":8593,"num_write":382,"num_write_kb":6103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"89'213","reported_seq":9591,"reported_epoch":91,"state":"remapped+peering","last_fresh":"2024-09-13T09:27:20.153054+0000","last_change":"2024-09-13T09:27:20.153054+0000","last_active":"2024-09-13T09:27:19.876926+0000","last_peered":"2024-09-13T09:27:18.814737+0000","last_clean":"2024-09-13T09:27:18.814737+0000","last_became_active":"2024-09-13T09:25:53.646396+0000","last_became_peered":"2024-09-13T09:25:53.646396+0000","last_unstale":"2024-09-13T09:27:20.153054+0000","last_undegraded":"2024-09-13T09:27:20.153054+0000","last_fullsized":"2024-09-13T09:27:20.153054+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:34:38.949042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":50919056,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":10,"num_read":9373,"num_read_kb":17821,"num_write":209,"num_write_kb":11320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"92'167","reported_seq":1828,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-09-13T09:27:21.551137+0000","last_change":"2024-09-13T09:27:21.551137+0000","last_active":"2024-09-13T09:27:21.551137+0000","last_peered":"2024-09-13T09:27:21.551137+0000","last_clean":"2024-09-13T09:27:18.308129+0000","last_became_active":"2024-09-13T09:27:21.181421+0000","last_became_peered":"2024-09-13T09:27:21.181421+0000","last_unstale":"2024-09-13T09:27:21.551137+0000","last_undegraded":"2024-09-13T09:27:21.551137+0000","last_fullsized":"2024-09-13T09:27:21.163170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:23:22.601877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":16418054,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1537,"num_read_kb":8936,"num_write":166,"num_write_kb":8932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1401001,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6],"avail_no_missing":["6","1","4"],"object_location_counts":[{"shards":"6","objects":17}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"89'274","reported_seq":3065,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:27:19.880631+0000","last_change":"2024-09-13T09:27:05.743332+0000","last_active":"2024-09-13T09:27:19.880631+0000","last_peered":"2024-09-13T09:27:19.880631+0000","last_clean":"2024-09-13T09:27:19.880631+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:27:19.880631+0000","last_undegraded":"2024-09-13T09:27:19.880631+0000","last_fullsized":"2024-09-13T09:27:19.880631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":64338478,"num_objects":54,"num_object_clones":12,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":9,"num_read":3022,"num_read_kb":23284,"num_write":353,"num_write_kb":18448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"89'404","reported_seq":6086,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.158384+0000","last_change":"2024-09-13T09:27:05.746491+0000","last_active":"2024-09-13T09:27:20.158384+0000","last_peered":"2024-09-13T09:27:20.158384+0000","last_clean":"2024-09-13T09:27:20.158384+0000","last_became_active":"2024-09-13T09:27:05.745595+0000","last_became_peered":"2024-09-13T09:27:05.745595+0000","last_unstale":"2024-09-13T09:27:20.158384+0000","last_undegraded":"2024-09-13T09:27:20.158384+0000","last_fullsized":"2024-09-13T09:27:20.158384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:38.773883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":40366412,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":3,"num_read":6644,"num_read_kb":20743,"num_write":615,"num_write_kb":14792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"89'237","reported_seq":13113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-13T09:27:20.156953+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:27:20.156953+0000","last_peered":"2024-09-13T09:27:20.156953+0000","last_clean":"2024-09-13T09:27:20.156953+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:27:20.156953+0000","last_undegraded":"2024-09-13T09:27:20.156953+0000","last_fullsized":"2024-09-13T09:27:20.156953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":77619749,"num_objects":49,"num_object_clones":13,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":11,"num_read":12896,"num_read_kb":32787,"num_write":267,"num_write_kb":27681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"89'939","reported_seq":6959,"reported_epoch":92,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:27:21.193964+0000","last_change":"2024-09-13T09:27:21.193964+0000","last_active":"2024-09-13T09:27:21.193964+0000","last_peered":"2024-09-13T09:27:21.193964+0000","last_clean":"2024-09-13T09:27:18.269642+0000","last_became_active":"2024-09-13T09:27:21.192691+0000","last_became_peered":"2024-09-13T09:27:21.192691+0000","last_unstale":"2024-09-13T09:27:21.193964+0000","last_undegraded":"2024-09-13T09:27:21.167731+0000","last_fullsized":"2024-09-13T09:27:21.159131+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:16.732239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":73169321,"num_objects":45,"num_object_clones":14,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":89,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":10,"num_read":10247,"num_read_kb":22801,"num_write":1574,"num_write_kb":16866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7],"avail_no_missing":["7","4"],"object_location_counts":[{"shards":"4,7","objects":45}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"88'231","reported_seq":5300,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-09-13T09:27:21.398120+0000","last_change":"2024-09-13T09:27:21.398479+0000","last_active":"2024-09-13T09:27:21.398120+0000","last_peered":"2024-09-13T09:27:21.398120+0000","last_clean":"2024-09-13T09:27:17.865862+0000","last_became_active":"2024-09-13T09:27:21.182673+0000","last_became_peered":"2024-09-13T09:27:21.182673+0000","last_unstale":"2024-09-13T09:27:21.398120+0000","last_undegraded":"2024-09-13T09:27:21.398120+0000","last_fullsized":"2024-09-13T09:27:21.159617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:28:31.230948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":57957507,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":6,"num_read":5014,"num_read_kb":28081,"num_write":268,"num_write_kb":12010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":76,"num_bytes_recovered":6443196,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7],"avail_no_missing":["7","0","4"],"object_location_counts":[{"shards":"7","objects":33}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"92'479","reported_seq":1792,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.437517+0000","last_change":"2024-09-13T09:26:42.574072+0000","last_active":"2024-09-13T09:27:21.437517+0000","last_peered":"2024-09-13T09:27:21.437517+0000","last_clean":"2024-09-13T09:27:21.437517+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:27:21.437517+0000","last_undegraded":"2024-09-13T09:27:21.437517+0000","last_fullsized":"2024-09-13T09:27:21.437517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":33658324,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":10,"num_read":2792,"num_read_kb":18241,"num_write":745,"num_write_kb":7349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"92'438","reported_seq":4662,"reported_epoch":92,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:27:21.583970+0000","last_change":"2024-09-13T09:27:21.194814+0000","last_active":"2024-09-13T09:27:21.583970+0000","last_peered":"2024-09-13T09:27:21.583970+0000","last_clean":"2024-09-13T09:27:17.865372+0000","last_became_active":"2024-09-13T09:27:21.194134+0000","last_became_peered":"2024-09-13T09:27:21.194134+0000","last_unstale":"2024-09-13T09:27:21.583970+0000","last_undegraded":"2024-09-13T09:27:21.162772+0000","last_fullsized":"2024-09-13T09:27:21.158665+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:41.261056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":45479478,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":5633,"num_read_kb":18841,"num_write":624,"num_write_kb":6059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7],"avail_no_missing":["7","4"],"object_location_counts":[{"shards":"4,7","objects":40},{"shards":"7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"84'214","reported_seq":4373,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-13T09:27:21.163286+0000","last_change":"2024-09-13T09:27:17.867086+0000","last_active":"2024-09-13T09:27:21.163286+0000","last_peered":"2024-09-13T09:27:21.163286+0000","last_clean":"2024-09-13T09:27:21.163286+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:27:21.163286+0000","last_undegraded":"2024-09-13T09:27:21.163286+0000","last_fullsized":"2024-09-13T09:27:21.163286+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45820774,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":4,"num_read":3792,"num_read_kb":18129,"num_write":272,"num_write_kb":11509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"92'524","reported_seq":5269,"reported_epoch":92,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:27:21.564496+0000","last_change":"2024-09-13T09:27:21.185823+0000","last_active":"2024-09-13T09:27:21.564496+0000","last_peered":"2024-09-13T09:27:21.564496+0000","last_clean":"2024-09-13T09:27:18.943086+0000","last_became_active":"2024-09-13T09:27:21.184794+0000","last_became_peered":"2024-09-13T09:27:21.184794+0000","last_unstale":"2024-09-13T09:27:21.564496+0000","last_undegraded":"2024-09-13T09:27:21.169417+0000","last_fullsized":"2024-09-13T09:27:21.158814+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:25:04.164850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":54769931,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":87,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":11,"num_read":6730,"num_read_kb":16426,"num_write":698,"num_write_kb":7823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7],"avail_no_missing":["7","4"],"object_location_counts":[{"shards":"4,7","objects":48},{"shards":"7","objects":2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1343463562,"num_objects":1188,"num_object_clones":255,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":216,"num_read":163657,"num_read_kb":520252,"num_write":14603,"num_write_kb":318450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1317,"num_bytes_recovered":100528528,"num_keys_recovered":340,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58392576,"data_stored":397367556,"data_compressed":31293652,"data_compressed_allocated":37195776,"data_compressed_original":380164109,"omap_allocated":8553,"internal_metadata":0},"log_size":11062,"ondisk_log_size":11062,"up":64,"acting":51,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55980,"kb_used_data":10908,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94315860,"statfs":{"total":96636764160,"available":96579440640,"internally_reserved":0,"allocated":11169792,"data_stored":70388730,"data_compressed":5577555,"data_compressed_allocated":5971968,"data_compressed_original":66237375,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.93400000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":8.2249999999999996,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.91600000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.626}]}]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38220,"kb_used_data":3004,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333620,"statfs":{"total":96636764160,"available":96597626880,"internally_reserved":0,"allocated":3076096,"data_stored":10877695,"data_compressed":983820,"data_compressed_allocated":1171456,"data_compressed_original":9779878,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.0339999999999998,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.093},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":5.149,"5min":5.149,"15min":5.149},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":4.9829999999999997,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":5.0709999999999997,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.012},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":6.2050000000000001,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.161}]}]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48852,"kb_used_data":9536,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94322988,"statfs":{"total":96636764160,"available":96586739712,"internally_reserved":0,"allocated":9764864,"data_stored":55594301,"data_compressed":4353304,"data_compressed_allocated":5259264,"data_compressed_original":52383498,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.89200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.0329999999999999}]}]},{"osd":4,"up_from":0,"seq":64424509464,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":74,"seq":317827579909,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41024,"kb_used_data":8120,"kb_used_omap":14,"kb_used_meta":32881,"kb_avail":94330816,"statfs":{"total":96636764160,"available":96594755584,"internally_reserved":0,"allocated":8314880,"data_stored":53354436,"data_compressed":3496164,"data_compressed_allocated":4571136,"data_compressed_original":50725972,"omap_allocated":14794,"internal_metadata":33670710},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]}]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44140,"kb_used_data":4824,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327700,"statfs":{"total":96636764160,"available":96591564800,"internally_reserved":0,"allocated":4939776,"data_stored":26002220,"data_compressed":1882339,"data_compressed_allocated":2568192,"data_compressed_original":24514963,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.995}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.115,"15min":1.115},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.071}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.034}]}]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49516,"kb_used_data":10200,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94322324,"statfs":{"total":96636764160,"available":96586059776,"internally_reserved":0,"allocated":10444800,"data_stored":68932652,"data_compressed":5468357,"data_compressed_allocated":6520832,"data_compressed_original":66421557,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.245},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.26800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":5.9420000000000002,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.45600000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.49199999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:26:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.222}]}]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38420,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333420,"statfs":{"total":96636764160,"available":96597422080,"internally_reserved":0,"allocated":3276800,"data_stored":15222010,"data_compressed":1221628,"data_compressed_allocated":1527808,"data_compressed_original":14173962,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.52100000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.72499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.0489999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:26:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.94899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":68956197,"data_compressed":5536072,"data_compressed_allocated":6598656,"data_compressed_original":66945845,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5087232,"data_stored":33540388,"data_compressed":2654168,"data_compressed_allocated":3477504,"data_compressed_original":32305555,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2207744,"data_stored":14727452,"data_compressed":1221628,"data_compressed_allocated":1527808,"data_compressed_original":14173962,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7274496,"data_stored":52873812,"data_compressed":3496164,"data_compressed_allocated":4571136,"data_compressed_original":50725972,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11247616,"data_stored":73747025,"data_compressed":5996228,"data_compressed_allocated":6799360,"data_compressed_original":69884103,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8785920,"data_stored":55626177,"data_compressed":4421019,"data_compressed_allocated":5337088,"data_compressed_original":52907786,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3350528,"data_stored":21471520,"data_compressed":1818081,"data_compressed_allocated":2228224,"data_compressed_original":20622423,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11001856,"data_stored":76424985,"data_compressed":6150292,"data_compressed_allocated":6656000,"data_compressed_original":72598463,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:27:25.912 INFO:tasks.thrashosds.thrasher:Setting 3.17 to [7, 6, 2, 5] 2024-09-13T09:27:25.912 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.17', '7', '6', '2', '5'] 2024-09-13T09:27:25.912 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.17 7 6 2 5 2024-09-13T09:27:25.915 INFO:teuthology.orchestra.run.smithi063.stdout:4043 write 0x150281d thru 0x150c5fa (0x9dde bytes) 2024-09-13T09:27:25.926 INFO:teuthology.orchestra.run.smithi063.stdout:4044 trunc from 0x539df5f to 0x722fb99 2024-09-13T09:27:25.933 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:25.935 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:25.936 INFO:teuthology.orchestra.run.smithi063.stdout:4045 read 0x6af0fbd thru 0x6af9fbd (0x9001 bytes) 2024-09-13T09:27:25.937 INFO:teuthology.orchestra.run.smithi063.stdout:4046 write 0x4a61a60 thru 0x4a6b675 (0x9c16 bytes) 2024-09-13T09:27:25.946 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.946 INFO:teuthology.orchestra.run.smithi063.stdout:4050 writesame 0x1dd23ee thru 0x1dde874 data_size 0x1(0xc487 bytes) 2024-09-13T09:27:25.968 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:25.968 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:25.975 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:25.975 INFO:teuthology.orchestra.run.smithi063.stdout:4055 read 0xbfcecb thru 0xbffea6 (0x2fdc bytes) 2024-09-13T09:27:25.978 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:25.982 INFO:teuthology.orchestra.run.smithi063.stdout:4056 trunc from 0x722fb99 to 0x2e8e400 2024-09-13T09:27:26.014 INFO:teuthology.orchestra.run.smithi063.stdout:4057 trunc from 0x2e8e400 to 0xaa69e1f 2024-09-13T09:27:26.014 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:26.007+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:26.024 INFO:teuthology.orchestra.run.smithi063.stdout:4058 punch from 0x272d806 to 0x272e785, (0xf7f bytes) 2024-09-13T09:27:26.024 INFO:teuthology.orchestra.run.smithi063.stdout:4059 read 0x56fed90 thru 0x570c366 (0xd5d7 bytes) 2024-09-13T09:27:26.024 INFO:teuthology.orchestra.run.smithi063.stdout:4060 write 0x8f0a257 thru 0x8f1059e (0x6348 bytes) 2024-09-13T09:27:26.031 INFO:teuthology.orchestra.run.smithi063.stdout:4062 read 0x92d1a6a thru 0x92e0e21 (0xf3b8 bytes) 2024-09-13T09:27:26.032 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.032 INFO:teuthology.orchestra.run.smithi063.stdout:4064 write 0x26a49d1 thru 0x26a6aa2 (0x20d2 bytes) 2024-09-13T09:27:26.038 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.038 INFO:teuthology.orchestra.run.smithi063.stdout:4066 write 0x7d25653 thru 0x7d2ec58 (0x9606 bytes) 2024-09-13T09:27:26.062 INFO:teuthology.orchestra.run.smithi063.stdout:4067 write 0xd0d9553 thru 0xd0e518a (0xbc38 bytes) 2024-09-13T09:27:26.068 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.068 INFO:teuthology.orchestra.run.smithi063.stdout:4069 punch from 0xc6c607 to 0xc72c04, (0x65fd bytes) 2024-09-13T09:27:26.069 INFO:teuthology.orchestra.run.smithi063.stdout:4072 write 0x8ecb976 thru 0x8ed9b3c (0xe1c7 bytes) 2024-09-13T09:27:26.085 INFO:teuthology.orchestra.run.smithi063.stdout:4073 trunc from 0xd0e518b to 0x6ffe4b1 2024-09-13T09:27:26.094 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:26.091+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:26.105 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.105 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.121 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:26.123 INFO:teuthology.orchestra.run.smithi063.stdout:4075 writesame 0xbb9dbd6 thru 0xbbab2ee data_size 0x5(0xd719 bytes) 2024-09-13T09:27:26.139 INFO:teuthology.orchestra.run.smithi063.stdout:4078 writesame 0xda66096 thru 0xda67ba4 data_size 0x1(0x1b0f bytes) 2024-09-13T09:27:26.139 INFO:teuthology.orchestra.run.smithi063.stdout:4079 writesame 0x1e6c30b thru 0x1e7ae38 data_size 0x1(0xeb2e bytes) 2024-09-13T09:27:26.190 INFO:teuthology.orchestra.run.smithi063.stdout:4080 write 0xe84ea38 thru 0xe85aa5a (0xc023 bytes) 2024-09-13T09:27:26.198 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:26.192+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:26.203 INFO:teuthology.orchestra.run.smithi063.stdout:4081 write 0x6ca8486 thru 0x6cac797 (0x4312 bytes) 2024-09-13T09:27:26.207 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:27:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:4082 punch from 0x71172f9 to 0x7121875, (0xa57c bytes) 2024-09-13T09:27:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:4083 writesame 0xe60a53 thru 0xe6c645 data_size 0x1(0xbbf3 bytes) 2024-09-13T09:27:26.237 INFO:teuthology.orchestra.run.smithi063.stdout:4084 read 0x5c824e9 thru 0x5c87268 (0x4d80 bytes) 2024-09-13T09:27:26.237 INFO:teuthology.orchestra.run.smithi063.stdout:4086 writesame 0x5a3e2b9 thru 0x5a44a41 data_size 0x5f(0x6789 bytes) 2024-09-13T09:27:26.252 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.252 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.261 INFO:teuthology.orchestra.run.smithi063.stdout:4089 trunc from 0xe85aa5b to 0x2a1c2f5 2024-09-13T09:27:26.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:4090 trunc from 0x2a1c2f5 to 0x25ea575 2024-09-13T09:27:26.304 INFO:teuthology.orchestra.run.smithi063.stdout:4091 trunc from 0x25ea575 to 0x816a4d0 2024-09-13T09:27:26.304 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:26.300+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:26.317 INFO:teuthology.orchestra.run.smithi063.stdout:4092 punch from 0x7df6b85 to 0x7e058c8, (0xed43 bytes) 2024-09-13T09:27:26.317 INFO:teuthology.orchestra.run.smithi063.stdout:4093 read 0x5a015cc thru 0x5a06c1d (0x5652 bytes) 2024-09-13T09:27:26.317 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.317 INFO:teuthology.orchestra.run.smithi063.stdout:4096 punch from 0x5883c3a to 0x588c7d7, (0x8b9d bytes) 2024-09-13T09:27:26.317 INFO:teuthology.orchestra.run.smithi063.stdout:4098 write 0x5be152e thru 0x5bf024c (0xed1f bytes) 2024-09-13T09:27:26.330 INFO:teuthology.orchestra.run.smithi063.stdout:4099 write 0x424857c thru 0x4251e21 (0x98a6 bytes) 2024-09-13T09:27:26.338 INFO:teuthology.orchestra.run.smithi063.stdout:4100 read 0x61efdb9 thru 0x61fec77 (0xeebf bytes) 2024-09-13T09:27:26.338 INFO:teuthology.orchestra.run.smithi063.stdout:4103 punch from 0x5b20a88 to 0x5b252a6, (0x481e bytes) 2024-09-13T09:27:26.339 INFO:teuthology.orchestra.run.smithi063.stdout:4104 writesame 0x5e4d7ff thru 0x5e54138 data_size 0x1(0x693a bytes) 2024-09-13T09:27:26.357 INFO:teuthology.orchestra.run.smithi063.stdout:4107 read 0x4811b74 thru 0x4816607 (0x4a94 bytes) 2024-09-13T09:27:26.359 INFO:teuthology.orchestra.run.smithi063.stdout:4109 trunc from 0x816a4d0 to 0x9b7d15d 2024-09-13T09:27:26.385 INFO:teuthology.orchestra.run.smithi063.stdout:4111 trunc from 0x9b7d15d to 0x138a3c6 2024-09-13T09:27:26.386 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.386 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.397 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:26.392+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:26.398 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:26.411 INFO:teuthology.orchestra.run.smithi063.stdout:4112 read 0x264ce7 thru 0x2725be (0xd8d8 bytes) 2024-09-13T09:27:26.412 INFO:teuthology.orchestra.run.smithi063.stdout:4113 read 0x73568 thru 0x7f5ec (0xc085 bytes) 2024-09-13T09:27:26.413 INFO:teuthology.orchestra.run.smithi063.stdout:4114 punch from 0x5fbf8e to 0x600ed8, (0x4f4a bytes) 2024-09-13T09:27:26.437 INFO:teuthology.orchestra.run.smithi063.stdout:4115 writesame 0x67affeb thru 0x67b8244 data_size 0xa(0x825a bytes) 2024-09-13T09:27:26.437 INFO:teuthology.orchestra.run.smithi063.stdout:4116 writesame 0x340cc29 thru 0x34144e1 data_size 0x7(0x78b9 bytes) 2024-09-13T09:27:26.448 INFO:teuthology.orchestra.run.smithi063.stdout:4118 read 0x1413a44 thru 0x1416acb (0x3088 bytes) 2024-09-13T09:27:26.474 INFO:teuthology.orchestra.run.smithi063.stdout:4119 write 0xe48bd8c thru 0xe48ddd2 (0x2047 bytes) 2024-09-13T09:27:26.480 INFO:teuthology.orchestra.run.smithi063.stdout:4120 read 0x37c4d41 thru 0x37cda64 (0x8d24 bytes) 2024-09-13T09:27:26.480 INFO:teuthology.orchestra.run.smithi063.stdout:4121 write 0xb2080f2 thru 0xb211722 (0x9631 bytes) 2024-09-13T09:27:26.489 INFO:teuthology.orchestra.run.smithi063.stdout:4122 writesame 0x64a6471 thru 0x64ae1d5 data_size 0x1(0x7d65 bytes) 2024-09-13T09:27:26.497 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:26.492+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:26.528 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.528 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.530 INFO:teuthology.orchestra.run.smithi063.stdout:4123 trunc from 0xe48ddd3 to 0x902c02a 2024-09-13T09:27:26.540 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:26.549 INFO:teuthology.orchestra.run.smithi063.stdout:4124 write 0x727b41d thru 0x7288dd1 (0xd9b5 bytes) 2024-09-13T09:27:26.572 INFO:teuthology.orchestra.run.smithi063.stdout:4125 trunc from 0x902c02a to 0x300510a 2024-09-13T09:27:26.595 INFO:teuthology.orchestra.run.smithi063.stdout:4127 read 0x8c4ba6 thru 0x8ce0a4 (0x94ff bytes) 2024-09-13T09:27:26.598 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:26.595+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:26.618 INFO:teuthology.orchestra.run.smithi063.stdout:4129 write 0x969ceae thru 0x96a4292 (0x73e5 bytes) 2024-09-13T09:27:26.625 INFO:teuthology.orchestra.run.smithi063.stdout:4130 punch from 0x155f3b3 to 0x156c4e7, (0xd134 bytes) 2024-09-13T09:27:26.626 INFO:teuthology.orchestra.run.smithi063.stdout:4131 read 0x125a177 thru 0x1264738 (0xa5c2 bytes) 2024-09-13T09:27:26.641 INFO:teuthology.orchestra.run.smithi063.stdout:4132 write 0xa5ccded thru 0xa5db105 (0xe319 bytes) 2024-09-13T09:27:26.654 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.654 INFO:teuthology.orchestra.run.smithi063.stdout:4134 read 0x1b0d0b3 thru 0x1b18a53 (0xb9a1 bytes) 2024-09-13T09:27:26.654 INFO:teuthology.orchestra.run.smithi063.stdout:4135 read 0x1d77899 thru 0x1d7c16f (0x48d7 bytes) 2024-09-13T09:27:26.654 INFO:teuthology.orchestra.run.smithi063.stdout:4137 read 0x6041ccd thru 0x6049518 (0x784c bytes) 2024-09-13T09:27:26.654 INFO:teuthology.orchestra.run.smithi063.stdout:4140 punch from 0x557741e to 0x5585ac8, (0xe6aa bytes) 2024-09-13T09:27:26.654 INFO:teuthology.orchestra.run.smithi063.stdout:4141 write 0x5c6c010 thru 0x5c7416a (0x815b bytes) 2024-09-13T09:27:26.661 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.661 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.661 INFO:teuthology.orchestra.run.smithi063.stdout:4144 writesame 0x670e6be thru 0x6711f53 data_size 0x1(0x3896 bytes) 2024-09-13T09:27:26.662 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.662 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.673 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:26.674 INFO:teuthology.orchestra.run.smithi063.stdout:4145 punch from 0x9db29aa to 0x9dbec63, (0xc2b9 bytes) 2024-09-13T09:27:26.674 INFO:teuthology.orchestra.run.smithi063.stdout:4146 writesame 0x49860dc thru 0x498634b data_size 0x8(0x270 bytes) 2024-09-13T09:27:26.678 INFO:teuthology.orchestra.run.smithi063.stdout:4147 write 0x2394058 thru 0x2398bf2 (0x4b9b bytes) 2024-09-13T09:27:26.691 INFO:teuthology.orchestra.run.smithi063.stdout:4148 trunc from 0xa5db106 to 0x676af27 2024-09-13T09:27:26.699 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:26.692+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:26.728 INFO:teuthology.orchestra.run.smithi063.stdout:4149 writesame 0x78e7ef2 thru 0x78ea247 data_size 0x1(0x2356 bytes) 2024-09-13T09:27:26.729 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.729 INFO:teuthology.orchestra.run.smithi063.stdout:4151 punch from 0x25f4a6a to 0x25f8fcc, (0x4562 bytes) 2024-09-13T09:27:26.729 INFO:teuthology.orchestra.run.smithi063.stdout:4152 read 0x2a8d6 thru 0x2d839 (0x2f64 bytes) 2024-09-13T09:27:26.740 INFO:teuthology.orchestra.run.smithi063.stdout:4154 trunc from 0x78ea248 to 0xca5cb49 2024-09-13T09:27:26.753 INFO:teuthology.orchestra.run.smithi063.stdout:4155 read 0xadea7e5 thru 0xadf1fc6 (0x77e2 bytes) 2024-09-13T09:27:26.753 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.770 INFO:teuthology.orchestra.run.smithi063.stdout:4157 write 0xe3caab6 thru 0xe3d7ff0 (0xd53b bytes) 2024-09-13T09:27:26.779 INFO:teuthology.orchestra.run.smithi063.stdout:4158 write 0x1b45746 thru 0x1b54428 (0xece3 bytes) 2024-09-13T09:27:26.789 INFO:teuthology.orchestra.run.smithi063.stdout:4160 read 0x3734d9f thru 0x373da52 (0x8cb4 bytes) 2024-09-13T09:27:26.789 INFO:teuthology.orchestra.run.smithi063.stdout:4161 read 0x3c91258 thru 0x3c9bf33 (0xacdc bytes) 2024-09-13T09:27:26.789 INFO:teuthology.orchestra.run.smithi063.stdout:4162 read 0x3e1f561 thru 0x3e26de4 (0x7884 bytes) 2024-09-13T09:27:26.789 INFO:teuthology.orchestra.run.smithi063.stdout:4163 writesame 0xbc6237e thru 0xbc65c3b data_size 0x1b(0x38be bytes) 2024-09-13T09:27:26.793 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.793 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.795 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.795 INFO:teuthology.orchestra.run.smithi063.stdout:4165 read 0xe24af30 thru 0xe24dd9b (0x2e6c bytes) 2024-09-13T09:27:26.795 INFO:teuthology.orchestra.run.smithi063.stdout:4166 write 0x3a26e10 thru 0x3a36cc9 (0xfeba bytes) 2024-09-13T09:27:26.800 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:26.796+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:26.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:26.805 INFO:teuthology.orchestra.run.smithi063.stdout:4167 write 0xa3105da thru 0xa313002 (0x2a29 bytes) 2024-09-13T09:27:26.814 INFO:teuthology.orchestra.run.smithi063.stdout:4169 trunc from 0xe3d7ff1 to 0xafe3b39 2024-09-13T09:27:26.834 INFO:teuthology.orchestra.run.smithi063.stdout:4170 punch from 0x83b3a19 to 0x83c1b6b, (0xe152 bytes) 2024-09-13T09:27:26.852 INFO:teuthology.orchestra.run.smithi063.stdout:4171 writesame 0xe066d28 thru 0xe070edb data_size 0x83(0xa1b4 bytes) 2024-09-13T09:27:26.866 INFO:teuthology.orchestra.run.smithi063.stdout:4173 trunc from 0xe070edc to 0x65c1b10 2024-09-13T09:27:26.901 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:26.896+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:26.909 INFO:teuthology.orchestra.run.smithi063.stdout:4174 write 0x7e15f25 thru 0x7e21021 (0xb0fd bytes) 2024-09-13T09:27:26.920 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:26.920 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:26.920 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.921 INFO:teuthology.orchestra.run.smithi063.stdout:4176 punch from 0x7d9969 to 0x7e1f8c, (0x8623 bytes) 2024-09-13T09:27:26.921 INFO:teuthology.orchestra.run.smithi063.stdout:4177 read 0x1136e2d thru 0x11404de (0x96b2 bytes) 2024-09-13T09:27:26.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:26.949 INFO:teuthology.orchestra.run.smithi063.stdout:4179 write 0xd15c9e0 thru 0xd16102b (0x464c bytes) 2024-09-13T09:27:26.956 INFO:teuthology.orchestra.run.smithi063.stdout:4182 read 0x80006a5 thru 0x80055c2 (0x4f1e bytes) 2024-09-13T09:27:26.956 INFO:teuthology.orchestra.run.smithi063.stdout:4183 punch from 0x64d97f8 to 0x64e1fbd, (0x87c5 bytes) 2024-09-13T09:27:26.956 INFO:teuthology.orchestra.run.smithi063.stdout:4185 write 0x5c49df6 thru 0x5c57306 (0xd511 bytes) 2024-09-13T09:27:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:4186 read 0x3009ad5 thru 0x300dfc4 (0x44f0 bytes) 2024-09-13T09:27:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:4187 read 0x8eaef5b thru 0x8eb4b00 (0x5ba6 bytes) 2024-09-13T09:27:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:4188 read 0xaf0ff6a thru 0xaf13f9c (0x4033 bytes) 2024-09-13T09:27:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:26.964 INFO:teuthology.orchestra.run.smithi063.stdout:4193 writesame 0x327b029 thru 0x32859cb data_size 0x1(0xa9a3 bytes) 2024-09-13T09:27:27.002 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:26.999+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:27.003 INFO:teuthology.orchestra.run.smithi063.stdout:4194 write 0xe96ee5b thru 0xe97e591 (0xf737 bytes) 2024-09-13T09:27:27.015 INFO:teuthology.orchestra.run.smithi063.stdout:4196 read 0xabeab80 thru 0xabfa957 (0xfdd8 bytes) 2024-09-13T09:27:27.016 INFO:teuthology.orchestra.run.smithi063.stdout:4197 writesame 0x5ad8ca4 thru 0x5ae3ad7 data_size 0x2(0xae34 bytes) 2024-09-13T09:27:27.040 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.040 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.045 INFO:teuthology.orchestra.run.smithi063.stdout:4198 writesame 0xecf14cb thru 0xecfbbcb data_size 0x1(0xa701 bytes) 2024-09-13T09:27:27.045 INFO:teuthology.orchestra.run.smithi063.stdout:4199 punch from 0xb8b5f00 to 0xb8bd0d4, (0x71d4 bytes) 2024-09-13T09:27:27.045 INFO:teuthology.orchestra.run.smithi063.stdout:4200 read 0x95ecf3e thru 0x95ee8e3 (0x19a6 bytes) 2024-09-13T09:27:27.045 INFO:teuthology.orchestra.run.smithi063.stdout:4201 punch from 0x14b83f1 to 0x14b9e94, (0x1aa3 bytes) 2024-09-13T09:27:27.045 INFO:teuthology.orchestra.run.smithi063.stdout:4202 write 0x609dc07 thru 0x60acf9d (0xf397 bytes) 2024-09-13T09:27:27.050 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:27.052 INFO:teuthology.orchestra.run.smithi063.stdout:4203 punch from 0x86c15b7 to 0x86c42db, (0x2d24 bytes) 2024-09-13T09:27:27.052 INFO:teuthology.orchestra.run.smithi063.stdout:4205 read 0xa295464 thru 0xa29a530 (0x50cd bytes) 2024-09-13T09:27:27.052 INFO:teuthology.orchestra.run.smithi063.stdout:4206 write 0x652abec thru 0x65335f6 (0x8a0b bytes) 2024-09-13T09:27:27.054 INFO:teuthology.orchestra.run.smithi063.stdout:4207 write 0x3d93c4d thru 0x3d95a59 (0x1e0d bytes) 2024-09-13T09:27:27.060 INFO:teuthology.orchestra.run.smithi063.stdout:4208 write 0x25be389 thru 0x25c1fab (0x3c23 bytes) 2024-09-13T09:27:27.066 INFO:teuthology.orchestra.run.smithi063.stdout:4209 read 0xb7eaeb1 thru 0xb7f83dc (0xd52c bytes) 2024-09-13T09:27:27.066 INFO:teuthology.orchestra.run.smithi063.stdout:4212 writesame 0x780a6b1 thru 0x781454c data_size 0x2(0x9e9c bytes) 2024-09-13T09:27:27.083 INFO:teuthology.orchestra.run.smithi063.stdout:4213 read 0x3c6e055 thru 0x3c775d7 (0x9583 bytes) 2024-09-13T09:27:27.083 INFO:teuthology.orchestra.run.smithi063.stdout:4214 read 0x2319813 thru 0x231f159 (0x5947 bytes) 2024-09-13T09:27:27.083 INFO:teuthology.orchestra.run.smithi063.stdout:4215 write 0xeef033 thru 0xef04f3 (0x14c1 bytes) 2024-09-13T09:27:27.093 INFO:teuthology.orchestra.run.smithi063.stdout:4217 trunc from 0xecfbbcc to 0xcb9fa62 2024-09-13T09:27:27.104 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:27.099+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:27.117 INFO:teuthology.orchestra.run.smithi063.stdout:4218 read 0x371e3f thru 0x371ee8 (0xaa bytes) 2024-09-13T09:27:27.118 INFO:teuthology.orchestra.run.smithi063.stdout:4219 punch from 0xc1c0c14 to 0xc1c71e5, (0x65d1 bytes) 2024-09-13T09:27:27.118 INFO:teuthology.orchestra.run.smithi063.stdout:4220 write 0xc205fe7 thru 0xc206836 (0x850 bytes) 2024-09-13T09:27:27.124 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.124 INFO:teuthology.orchestra.run.smithi063.stdout:4224 write 0x5db578a thru 0x5db67a7 (0x101e bytes) 2024-09-13T09:27:27.128 INFO:teuthology.orchestra.run.smithi063.stdout:4227 read 0x8a9c1d9 thru 0x8a9f97e (0x37a6 bytes) 2024-09-13T09:27:27.128 INFO:teuthology.orchestra.run.smithi063.stdout:4228 read 0x9dafd0b thru 0x9db6ffd (0x72f3 bytes) 2024-09-13T09:27:27.128 INFO:teuthology.orchestra.run.smithi063.stdout:4229 read 0x579d910 thru 0x579eade (0x11cf bytes) 2024-09-13T09:27:27.128 INFO:teuthology.orchestra.run.smithi063.stdout:4230 write 0x47f5bd4 thru 0x47f99c0 (0x3ded bytes) 2024-09-13T09:27:27.138 INFO:teuthology.orchestra.run.smithi063.stdout:4231 read 0x9cd4d33 thru 0x9cd57dc (0xaaa bytes) 2024-09-13T09:27:27.138 INFO:teuthology.orchestra.run.smithi063.stdout:4233 read 0x44f21d6 thru 0x44fdbef (0xba1a bytes) 2024-09-13T09:27:27.138 INFO:teuthology.orchestra.run.smithi063.stdout:4234 read 0x86c338f thru 0x86ca9c1 (0x7633 bytes) 2024-09-13T09:27:27.138 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.138 INFO:teuthology.orchestra.run.smithi063.stdout:4240 write 0xa06af0b thru 0xa070661 (0x5757 bytes) 2024-09-13T09:27:27.155 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.155 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.158 INFO:teuthology.orchestra.run.smithi063.stdout:4242 trunc from 0xcb9fa62 to 0x7282a4b 2024-09-13T09:27:27.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:27.181 INFO:teuthology.orchestra.run.smithi063.stdout:4243 writesame 0x851b07 thru 0x85799f data_size 0x1(0x5e99 bytes) 2024-09-13T09:27:27.212 INFO:teuthology.orchestra.run.smithi063.stdout:4245 write 0x7b2b614 thru 0x7b32a60 (0x744d bytes) 2024-09-13T09:27:27.214 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:27.211+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:27.227 INFO:teuthology.orchestra.run.smithi063.stderr:set 3.17 pg_upmap_items mapping to [7->6,2->5] 2024-09-13T09:27:27.227 INFO:teuthology.orchestra.run.smithi063.stdout:4246 read 0x5c76503 thru 0x5c78a7a (0x2578 bytes) 2024-09-13T09:27:27.228 INFO:teuthology.orchestra.run.smithi063.stdout:4247 write 0x3d67ce9 thru 0x3d77a37 (0xfd4f bytes) 2024-09-13T09:27:27.236 INFO:teuthology.orchestra.run.smithi063.stdout:4249 writesame 0x7d8cd5 thru 0x7e6d9e data_size 0x45(0xe0ca bytes) 2024-09-13T09:27:27.260 INFO:teuthology.orchestra.run.smithi063.stdout:4251 writesame 0xc423e65 thru 0xc43277e data_size 0x1(0xe91a bytes) 2024-09-13T09:27:27.260 INFO:teuthology.orchestra.run.smithi063.stdout:4252 read 0x7317a73 thru 0x731955d (0x1aeb bytes) 2024-09-13T09:27:27.260 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.260 INFO:teuthology.orchestra.run.smithi063.stdout:4254 read 0x4132065 thru 0x4133a36 (0x19d2 bytes) 2024-09-13T09:27:27.260 INFO:teuthology.orchestra.run.smithi063.stdout:4256 read 0x3387d85 thru 0x3388f23 (0x119f bytes) 2024-09-13T09:27:27.276 INFO:teuthology.orchestra.run.smithi063.stdout:4258 write 0xe66b581 thru 0xe67a843 (0xf2c3 bytes) 2024-09-13T09:27:27.288 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.305 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:27.299+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:27.315 INFO:teuthology.orchestra.run.smithi063.stdout:4260 trunc from 0xe67a844 to 0x9b0371 2024-09-13T09:27:27.324 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.325 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.336 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:27.351 INFO:teuthology.orchestra.run.smithi063.stdout:4261 punch from 0x4b519d to 0x4b6d0b, (0x1b6e bytes) 2024-09-13T09:27:27.352 INFO:teuthology.orchestra.run.smithi063.stdout:4263 punch from 0x431d4b to 0x4344e9, (0x279e bytes) 2024-09-13T09:27:27.386 INFO:teuthology.orchestra.run.smithi063.stdout:4265 write 0xcaae208 thru 0xcab38e7 (0x56e0 bytes) 2024-09-13T09:27:27.394 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.394 INFO:teuthology.orchestra.run.smithi063.stdout:4267 punch from 0x820a847 to 0x82191b4, (0xe96d bytes) 2024-09-13T09:27:27.395 INFO:teuthology.orchestra.run.smithi063.stdout:4268 punch from 0xad8e30b to 0xad9a498, (0xc18d bytes) 2024-09-13T09:27:27.395 INFO:teuthology.orchestra.run.smithi063.stdout:4269 read 0xc07b994 thru 0xc08af50 (0xf5bd bytes) 2024-09-13T09:27:27.396 INFO:teuthology.orchestra.run.smithi063.stdout:4270 trunc from 0xcab38e8 to 0xdabb079 2024-09-13T09:27:27.406 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:27.400+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:27.409 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.409 INFO:teuthology.orchestra.run.smithi063.stdout:4272 read 0xcb69692 thru 0xcb6de7d (0x47ec bytes) 2024-09-13T09:27:27.409 INFO:teuthology.orchestra.run.smithi063.stdout:4273 write 0x38264a1 thru 0x382682b (0x38b bytes) 2024-09-13T09:27:27.415 INFO:teuthology.orchestra.run.smithi063.stdout:4274 write 0x354707b thru 0x35510c1 (0xa047 bytes) 2024-09-13T09:27:27.437 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.437 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.440 INFO:teuthology.orchestra.run.smithi063.stdout:4275 trunc from 0xdabb079 to 0x6308ca3 2024-09-13T09:27:27.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:27.462 INFO:teuthology.orchestra.run.smithi063.stdout:4276 read 0x183fbbc thru 0x184518f (0x55d4 bytes) 2024-09-13T09:27:27.467 INFO:teuthology.orchestra.run.smithi063.stdout:4277 trunc from 0x6308ca3 to 0x393e6a5 2024-09-13T09:27:27.483 INFO:teuthology.orchestra.run.smithi063.stdout:4278 read 0x1f6f2e4 thru 0x1f7bf46 (0xcc63 bytes) 2024-09-13T09:27:27.483 INFO:teuthology.orchestra.run.smithi063.stdout:4279 read 0x2e4e265 thru 0x2e5b1f6 (0xcf92 bytes) 2024-09-13T09:27:27.483 INFO:teuthology.orchestra.run.smithi063.stdout:4280 read 0xfe7e7d thru 0xfeb1d8 (0x335c bytes) 2024-09-13T09:27:27.507 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:27.503+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:27.516 INFO:teuthology.orchestra.run.smithi063.stdout:4281 write 0xde86aae thru 0xde8a88f (0x3de2 bytes) 2024-09-13T09:27:27.526 INFO:teuthology.orchestra.run.smithi063.stdout:4282 writesame 0x599683f thru 0x59a400c data_size 0x17(0xd7ce bytes) 2024-09-13T09:27:27.536 INFO:teuthology.orchestra.run.smithi063.stdout:4283 read 0xc0789d thru 0xc0e1b2 (0x6916 bytes) 2024-09-13T09:27:27.538 INFO:teuthology.orchestra.run.smithi063.stdout:4284 writesame 0x6b6a4a3 thru 0x6b73a73 data_size 0x1(0x95d1 bytes) 2024-09-13T09:27:27.559 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.559 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.559 INFO:teuthology.orchestra.run.smithi063.stdout:4287 writesame 0x4dda090 thru 0x4de7479 data_size 0x23(0xd3ea bytes) 2024-09-13T09:27:27.560 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.560 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.567 INFO:teuthology.orchestra.run.smithi063.stdout:4289 punch from 0x3b783d9 to 0x3b7fc47, (0x786e bytes) 2024-09-13T09:27:27.567 INFO:teuthology.orchestra.run.smithi063.stdout:4292 read 0x657db8c thru 0x658129e (0x3713 bytes) 2024-09-13T09:27:27.567 INFO:teuthology.orchestra.run.smithi063.stdout:4293 punch from 0x84d115b to 0x84e044a, (0xf2ef bytes) 2024-09-13T09:27:27.567 INFO:teuthology.orchestra.run.smithi063.stdout:4294 write 0xa0453f3 thru 0xa04696f (0x157d bytes) 2024-09-13T09:27:27.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:27.572 INFO:teuthology.orchestra.run.smithi063.stdout:4295 flatten 2024-09-13T09:27:27.583 INFO:teuthology.orchestra.run.smithi063.stdout:4298 read 0x5849c32 thru 0x584e811 (0x4be0 bytes) 2024-09-13T09:27:27.583 INFO:teuthology.orchestra.run.smithi063.stdout:4299 punch from 0x606050b to 0x606af6e, (0xaa63 bytes) 2024-09-13T09:27:27.583 INFO:teuthology.orchestra.run.smithi063.stdout:4300 write 0x8e90776 thru 0x8e98bed (0x8478 bytes) 2024-09-13T09:27:27.590 INFO:teuthology.orchestra.run.smithi063.stdout:4302 writesame 0x5ea9165 thru 0x5ead55b data_size 0x1(0x43f7 bytes) 2024-09-13T09:27:27.607 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:27.603+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:27.609 INFO:teuthology.orchestra.run.smithi063.stdout:4303 read 0x985f99b thru 0x9865e50 (0x64b6 bytes) 2024-09-13T09:27:27.609 INFO:teuthology.orchestra.run.smithi063.stdout:4304 writesame 0x60fd3fe thru 0x6104925 data_size 0x2e(0x7528 bytes) 2024-09-13T09:27:27.616 INFO:teuthology.orchestra.run.smithi063.stdout:4306 read 0x7e1211b thru 0x7e186b6 (0x659c bytes) 2024-09-13T09:27:27.617 INFO:teuthology.orchestra.run.smithi063.stdout:4308 writesame 0x48537eb thru 0x4853824 data_size 0x1(0x3a bytes) 2024-09-13T09:27:27.623 INFO:teuthology.orchestra.run.smithi063.stdout:4311 writesame 0x4133843 thru 0x413fb2c data_size 0x3d(0xc2ea bytes) 2024-09-13T09:27:27.646 INFO:teuthology.orchestra.run.smithi063.stdout:4312 writesame 0xe8ffe48 thru 0xe906046 data_size 0x1(0x61ff bytes) 2024-09-13T09:27:27.659 INFO:teuthology.orchestra.run.smithi063.stdout:4314 trunc from 0xe906047 to 0x8ee4e44 2024-09-13T09:27:27.675 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.675 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:27.696 INFO:teuthology.orchestra.run.smithi063.stdout:4316 write 0x9794b1f thru 0x979b1ed (0x66cf bytes) 2024-09-13T09:27:27.706 INFO:teuthology.orchestra.run.smithi063.stdout:4317 writesame 0x1cec766 thru 0x1cfb242 data_size 0x7d(0xeadd bytes) 2024-09-13T09:27:27.708 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:27.704+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:27.716 INFO:teuthology.orchestra.run.smithi063.stdout:4318 read 0x922a583 thru 0x9234c1c (0xa69a bytes) 2024-09-13T09:27:27.716 INFO:teuthology.orchestra.run.smithi063.stdout:4319 read 0x29dcc67 thru 0x29e81ab (0xb545 bytes) 2024-09-13T09:27:27.716 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.716 INFO:teuthology.orchestra.run.smithi063.stdout:4321 punch from 0x181a885 to 0x1823a9b, (0x9216 bytes) 2024-09-13T09:27:27.722 INFO:teuthology.orchestra.run.smithi063.stdout:4322 trunc from 0x979b1ee to 0xc05956d 2024-09-13T09:27:27.742 INFO:teuthology.orchestra.run.smithi063.stdout:4324 trunc from 0xc05956d to 0xe8db258 2024-09-13T09:27:27.753 INFO:teuthology.orchestra.run.smithi063.stdout:4325 write 0x8b157d0 thru 0x8b1c904 (0x7135 bytes) 2024-09-13T09:27:27.763 INFO:teuthology.orchestra.run.smithi063.stdout:4327 writesame 0xc930a51 thru 0xc93f542 data_size 0x3d(0xeaf2 bytes) 2024-09-13T09:27:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:4328 read 0x3575a8f thru 0x358266e (0xcbe0 bytes) 2024-09-13T09:27:27.772 INFO:teuthology.orchestra.run.smithi063.stdout:4329 writesame 0x12499c0 thru 0x124fc74 data_size 0x1(0x62b5 bytes) 2024-09-13T09:27:27.793 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.793 INFO:teuthology.orchestra.run.smithi063.stdout:4332 write 0xaee2b9 thru 0xaf1df6 (0x3b3e bytes) 2024-09-13T09:27:27.795 INFO:teuthology.orchestra.run.smithi063.stdout:4333 write 0x4e9890a thru 0x4ea8243 (0xf93a bytes) 2024-09-13T09:27:27.809 INFO:teuthology.orchestra.run.smithi063.stdout:4334 trunc from 0xe8db258 to 0xd241019 2024-09-13T09:27:27.809 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:27.803+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:27.821 INFO:teuthology.orchestra.run.smithi063.stdout:4335 punch from 0x661bb48 to 0x66296d4, (0xdb8c bytes) 2024-09-13T09:27:27.821 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.821 INFO:teuthology.orchestra.run.smithi063.stdout:4340 punch from 0x430fc08 to 0x4313301, (0x36f9 bytes) 2024-09-13T09:27:27.821 INFO:teuthology.orchestra.run.smithi063.stdout:4341 writesame 0x8318fe7 thru 0x8324a3f data_size 0x2f(0xba59 bytes) 2024-09-13T09:27:27.829 INFO:teuthology.orchestra.run.smithi063.stdout:4344 punch from 0xf0e644 to 0xf15b22, (0x74de bytes) 2024-09-13T09:27:27.830 INFO:teuthology.orchestra.run.smithi063.stdout:4345 writesame 0xa25c4d0 thru 0xa265c8b data_size 0x9c(0x97bc bytes) 2024-09-13T09:27:27.843 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.843 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.853 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:27.864 INFO:teuthology.orchestra.run.smithi063.stdout:4347 trunc from 0xd241019 to 0xe01ae8 2024-09-13T09:27:27.896 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.896 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:27.896 INFO:teuthology.orchestra.run.smithi063.stdout:4350 clone 33 order 19 su 32768 sc 5 2024-09-13T09:27:27.910 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:27.904+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:27.951 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:27.964 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:27.965 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:27.974 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:28.011 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:28.007+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:28.078 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:28.079 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:28.088 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:28.106 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:28.112 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:28.107+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:28.128 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:28.147 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:28.167 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:28.188 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:28.192 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:28.192 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:28.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:28.210 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:28.226 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:27:28.237 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:28.249 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:28.247+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:28.260 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:28.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:28.314 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:28.308+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:28.369 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.405563378, 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.292343633999998, 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:28.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:28.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.384 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.291945619000003, 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:28.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:28.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:28.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:28.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:28.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.392 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616838743000002, 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.393 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:28.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:28.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616823811000003, 2024-09-13T09:27:28.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:28.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:28.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:28.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:28.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.403 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.404 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.404 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:28.404 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616790705, 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:28.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:28.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:28.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:28.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:28.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:28.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:28.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616774411999998, 2024-09-13T09:27:28.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:28.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:28.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616758296999997, 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:28.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:28.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:28.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:28.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:28.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616727255999997, 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:28.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.428 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.616695632999999, 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:28.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86976 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 745472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.605802320999999, 2024-09-13T09:27:28.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054454226000000001, 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86976 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760776+0000", 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760775+0000", 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760778+0000", 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760784+0000", 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760789+0000", 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5399999999999997e-06 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815065+0000", 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05427684 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815134+0000", 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8224000000000003e-05 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815230+0000", 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6268999999999996e-05 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.438 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.605781984000004, 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:28.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:27:28.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603747572000003, 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:28.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603730515999999, 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:28.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:28.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:28.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:28.455 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603714347999997, 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:28.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603500685999997, 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603481375000001, 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603464981999998, 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603449320000003, 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.603422965999997, 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:28.488 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:28.489 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T09:27:28.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:28.490 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:28.484+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:28.590 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:28.587+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:28.617 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:28.617 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:28.617 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:28.617 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:28.627 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:28.691 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:28.684+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:28.727 INFO:teuthology.orchestra.run.smithi063.stdout:4352 punch from 0x8481b8 to 0x8548d0, (0xc718 bytes) 2024-09-13T09:27:28.735 INFO:teuthology.orchestra.run.smithi063.stdout:4353 read 0xc9918e thru 0xca87b3 (0xf626 bytes) 2024-09-13T09:27:28.735 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:28.735 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:28.736 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:28.736 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:28.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:28.765 INFO:teuthology.orchestra.run.smithi063.stdout:4354 writesame 0xb74f890 thru 0xb75b793 data_size 0x4b(0xbf04 bytes) 2024-09-13T09:27:28.765 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:28.765 INFO:teuthology.orchestra.run.smithi063.stdout:4357 punch from 0x9068ab5 to 0x9075d8d, (0xd2d8 bytes) 2024-09-13T09:27:28.765 INFO:teuthology.orchestra.run.smithi063.stdout:4358 write 0x41b26ea thru 0x41befe2 (0xc8f9 bytes) 2024-09-13T09:27:28.773 INFO:teuthology.orchestra.run.smithi063.stdout:4359 writesame 0x34f6e7c thru 0x34ff291 data_size 0x3a(0x8416 bytes) 2024-09-13T09:27:28.791 INFO:teuthology.orchestra.run.smithi063.stdout:4360 trunc from 0xb75b794 to 0x5ec33e1 2024-09-13T09:27:28.792 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:28.787+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:28.803 INFO:teuthology.orchestra.run.smithi063.stdout:4361 punch from 0x136bd44 to 0x137261a, (0x68d6 bytes) 2024-09-13T09:27:28.803 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:28.803 INFO:teuthology.orchestra.run.smithi063.stdout:4363 read 0x67ca9d thru 0x68144b (0x49af bytes) 2024-09-13T09:27:28.829 INFO:teuthology.orchestra.run.smithi063.stdout:4364 write 0xab58c9e thru 0xab5e3e7 (0x574a bytes) 2024-09-13T09:27:28.835 INFO:teuthology.orchestra.run.smithi063.stdout:4365 punch from 0x2727436 to 0x272a3f7, (0x2fc1 bytes) 2024-09-13T09:27:28.836 INFO:teuthology.orchestra.run.smithi063.stdout:4366 read 0x4558d56 thru 0x4559e9d (0x1148 bytes) 2024-09-13T09:27:28.836 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:28.836 INFO:teuthology.orchestra.run.smithi063.stdout:4368 write 0x85e41bb thru 0x85e7244 (0x308a bytes) 2024-09-13T09:27:28.841 INFO:teuthology.orchestra.run.smithi063.stdout:4369 write 0x3c28c34 thru 0x3c2cf69 (0x4336 bytes) 2024-09-13T09:27:28.848 INFO:teuthology.orchestra.run.smithi063.stdout:4370 read 0x6c59c7d thru 0x6c5f3ba (0x573e bytes) 2024-09-13T09:27:28.848 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:28.869 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:28.869 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:28.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:28.869 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:28.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.869 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.126635613000001, 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:28.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:28.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:28.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124576 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 0~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:28.876 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.730130056, 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050917829999999999, 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124576 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134159+0000", 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134158+0000", 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134164+0000", 2024-09-13T09:27:28.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.083e-06 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134180+0000", 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5177000000000001e-05 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.134192+0000", 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.216e-05 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138437+0000", 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042449430000000002 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.138535+0000", 2024-09-13T09:27:28.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7962999999999997e-05 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139251+0000", 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071628499999999999 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124584 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.728959496000002, 2024-09-13T09:27:28.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.004962073, 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124584 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135330+0000", 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135327+0000", 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135341+0000", 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3454e-05 2024-09-13T09:27:28.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135366+0000", 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.51e-05 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135378+0000", 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2048000000000001e-05 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139290+0000", 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039123580000000003 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.139394+0000", 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010412999999999999 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140292+0000", 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00089737400000000004 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.885 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124586 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.728899633000001, 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062408630000000001, 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124586 2024-09-13T09:27:28.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135389+0000", 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135396+0000", 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7309999999999996e-06 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135404+0000", 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5369999999999997e-06 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135411+0000", 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140323+0000", 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049127659999999998 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140390+0000", 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6631999999999995e-05 2024-09-13T09:27:28.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141630+0000", 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012403480000000001 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.890 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.893 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124592 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.893 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:28.893 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.728825805, 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0076439699999999999, 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124592 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:28.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135469+0000", 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135500+0000", 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1149000000000003e-05 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135513+0000", 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2771e-05 2024-09-13T09:27:28.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142205+0000", 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066924239999999998 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142329+0000", 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012366100000000001 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143107+0000", 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077859999999999995 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124596 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.728713995, 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078700200000000001, 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124596 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:28.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135574+0000", 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135581+0000", 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3280000000000002e-06 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135597+0000", 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5996999999999999e-05 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135608+0000", 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.166e-05 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143162+0000", 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0075534349999999998 2024-09-13T09:27:28.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143306+0000", 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014448900000000001 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143445+0000", 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013898200000000001 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124608 3.7 3:e4e5f482:::rbd_data.1118f3eff690.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.719236079000002, 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062754919999999997, 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124608 2024-09-13T09:27:28.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145052+0000", 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145531+0000", 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00047855199999999998 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145542+0000", 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0961000000000001e-05 2024-09-13T09:27:28.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145551+0000", 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145661+0000", 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109726 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145992+0000", 2024-09-13T09:27:28.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033164000000000002 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.146845+0000", 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085247199999999995 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.148693+0000", 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018483460000000001 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151235+0000", 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002542159 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151297+0000", 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1710000000000004e-05 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151329+0000", 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1730000000000003e-05 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.908 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124700 3.1e 3:78e395d8:::rbd_data.1118f3eff690.0000000000000080:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.561992883999999, 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011658599, 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.909 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124700 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302294+0000", 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302307+0000", 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3602e-05 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302336+0000", 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8116999999999999e-05 2024-09-13T09:27:28.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302348+0000", 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2792000000000001e-05 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302394+0000", 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5797000000000002e-05 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302505+0000", 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111286 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.304334+0000", 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828325 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313641+0000", 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093073449999999999 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313729+0000", 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8305000000000002e-05 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313834+0000", 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010443899999999999 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313955+0000", 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121142 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:124700 3.1e e78/77 ondisk, result = 0)", 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.556108146, 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057895580000000002, 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308180+0000", 2024-09-13T09:27:28.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308188+0000", 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308203+0000", 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4977000000000001e-05 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308216+0000", 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2855e-05 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313698+0000", 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054822960000000002 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313716+0000", 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8167000000000001e-05 2024-09-13T09:27:28.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313971+0000", 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025462500000000002 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125238 3.18 3:1cc4019a:::rbd_data.1118bda513e1.0000000000000005:head [stat,write 16509626~3298] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.822969388000001, 2024-09-13T09:27:28.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065355600000000002, 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125238 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041315+0000", 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999957 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041336+0000", 2024-09-13T09:27:28.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1073e-05 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041373+0000", 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7039000000000002e-05 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041394+0000", 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0477000000000001e-05 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041475+0000", 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0952999999999996e-05 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041481+0000", 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045492+0000", 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040115339999999998 2024-09-13T09:27:28.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045585+0000", 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2255000000000004e-05 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045728+0000", 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014281499999999999 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047526+0000", 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017983400000000001 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047681+0000", 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015514200000000001 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047777+0000", 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6415000000000004e-05 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047855+0000", 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7985999999999995e-05 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126514 3.1e 3:7f6180ef:::rbd_object_map.1118413dc0c1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.637291461, 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0077678299999999999, 2024-09-13T09:27:28.926 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126514 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226997+0000", 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227004+0000", 2024-09-13T09:27:28.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.187e-06 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227027+0000", 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3254000000000001e-05 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227034+0000", 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6479999999999997e-06 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227078+0000", 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227182+0000", 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104095 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:28.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227687+0000", 2024-09-13T09:27:28.931 INFO:teuthology.orchestra.run.smithi063.stdout:4372 write 0xec7c4c0 thru 0xec80a02 (0x4543 bytes) 2024-09-13T09:27:28.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00050565500000000004 2024-09-13T09:27:28.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.233076+0000", 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053885319999999997 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234572+0000", 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014962490000000001 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234697+0000", 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000124626 2024-09-13T09:27:28.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234766+0000", 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8842000000000003e-05 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126516 3.1e 3:7f6180ef:::rbd_object_map.1118413dc0c1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.636914482, 2024-09-13T09:27:28.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078524679999999996, 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126516 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227373+0000", 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227386+0000", 2024-09-13T09:27:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2916e-05 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227409+0000", 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3268000000000002e-05 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227425+0000", 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5798e-05 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.230833+0000", 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034084919999999999 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.231022+0000", 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000188763 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.231925+0000", 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00090269899999999997 2024-09-13T09:27:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234795+0000", 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028700589999999999 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235100+0000", 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030537400000000002 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235153+0000", 2024-09-13T09:27:28.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2222999999999998e-05 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235227+0000", 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4719000000000001e-05 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:127527 3.8 3:149f3857:::rbd_header.11181a204f0d:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:19.878053+0000", 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.9862365060000009, 2024-09-13T09:27:28.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050577039999999997, 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 127527 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878053+0000", 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878053+0000", 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878049+0000", 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878067+0000", 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8119999999999999e-05 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878100+0000", 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3108000000000002e-05 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878117+0000", 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6197000000000001e-05 2024-09-13T09:27:28.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878195+0000", 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8423000000000005e-05 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.878201+0000", 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8409999999999996e-06 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.880505+0000", 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023038310000000001 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.880649+0000", 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014465100000000001 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.881167+0000", 2024-09-13T09:27:28.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00051791999999999995 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.882310+0000", 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001142426 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.883018+0000", 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00070869200000000002 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.883072+0000", 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3570999999999999e-05 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:19.883110+0000", 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8491999999999997e-05 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.946 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128801 3.7 3:e5fee451:::rbd_data.11181500dc3e.0000000000000008:head [stat,write 32768~14443] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.622455+0000", 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.2418341179999999, 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044694649999999997, 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128801 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622455+0000", 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622455+0000", 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622453+0000", 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:28.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622473+0000", 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9507000000000001e-05 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622494+0000", 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1455999999999999e-05 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622504+0000", 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5510000000000005e-06 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622569+0000", 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5289000000000005e-05 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622653+0000", 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3283999999999997e-05 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622883+0000", 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023047700000000001 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.624457+0000", 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015737609999999999 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626780+0000", 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023235909999999999 2024-09-13T09:27:28.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626842+0000", 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1818999999999999e-05 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626925+0000", 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2368999999999997e-05 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128961 3.b 3:d06f0007:::rbd_data.11181500dc3e.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.0170226639999997, 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010200722000000001, 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128961 2024-09-13T09:27:28.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847265+0000", 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847762+0000", 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00049738799999999998 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847787+0000", 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4451999999999999e-05 2024-09-13T09:27:28.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847799+0000", 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2140999999999999e-05 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847886+0000", 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7020000000000004e-05 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.848076+0000", 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019032700000000001 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852181+0000", 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041049830000000004 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857348+0000", 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051670809999999996 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857367+0000", 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8533e-05 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857428+0000", 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1334999999999995e-05 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857467+0000", 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9286000000000003e-05 2024-09-13T09:27:28.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128963 3.1e 3:7a2537bd:::rbd_data.11181500dc3e.000000000000005b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888542+0000", 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.9757475859999998, 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049817630000000002, 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.959 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128963 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888542+0000", 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888542+0000", 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888540+0000", 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888553+0000", 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3329999999999999e-05 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888586+0000", 2024-09-13T09:27:28.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2925000000000001e-05 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888603+0000", 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6730000000000001e-05 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888762+0000", 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015963800000000001 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888978+0000", 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000216176 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889216+0000", 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023740299999999999 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.891435+0000", 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002219412 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893451+0000", 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020155199999999998 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893488+0000", 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7635999999999997e-05 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893523+0000", 2024-09-13T09:27:28.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5064000000000002e-05 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129048 3.b 3:d1c06694:::rbd_data.11181500dc3e.0000000000000088:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8219861709999998, 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014253025000000001, 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129048 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042302+0000", 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042309+0000", 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.601e-06 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042322+0000", 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3529999999999999e-05 2024-09-13T09:27:28.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042327+0000", 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042399+0000", 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1208e-05 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042524+0000", 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012532799999999999 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.044815+0000", 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022907729999999999 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056394+0000", 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011579271 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056445+0000", 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0994e-05 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056500+0000", 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5442999999999998e-05 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056556+0000", 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5702000000000003e-05 2024-09-13T09:27:28.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129113 3.7 3:e0643571:::rbd_data.11181500dc3e.000000000000016e:head [set-alloc-hint object_size 524288 write_size 524288,write 98304~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.7309380550000002, 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057129440000000002, 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129113 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:28.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133348+0000", 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133380+0000", 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2150000000000002e-05 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133409+0000", 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8924e-05 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133423+0000", 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3773e-05 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133527+0000", 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010395600000000001 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133780+0000", 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025285500000000002 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134012+0000", 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023174799999999999 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.137751+0000", 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037395369999999998 2024-09-13T09:27:28.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.138980+0000", 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001228282 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.139035+0000", 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5652999999999997e-05 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.139064+0000", 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8969e-05 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129140 3.7 3:e0643571:::rbd_data.11181500dc3e.000000000000016e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.6529895290000001, 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086215130000000008, 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.977 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129140 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:28.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211298+0000", 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211307+0000", 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6829999999999997e-06 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212397+0000", 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010900009999999999 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212409+0000", 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2746e-05 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212490+0000", 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0668999999999996e-05 2024-09-13T09:27:28.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.213419+0000", 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092909700000000004 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.213568+0000", 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000148531 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.214326+0000", 2024-09-13T09:27:28.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00075828500000000004 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219843+0000", 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055167430000000002 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219875+0000", 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1869000000000003e-05 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219921+0000", 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6560000000000001e-05 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130684 3.7 3:e1961e3b:::rbd_data.1118de6ae5af.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.812071+0000", 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.052218641000000003, 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055824189999999999, 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:28.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130684 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812071+0000", 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812071+0000", 2024-09-13T09:27:28.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812068+0000", 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812423+0000", 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00035447799999999998 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812448+0000", 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5055e-05 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812459+0000", 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1691e-05 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812587+0000", 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012760399999999999 2024-09-13T09:27:28.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.813005+0000", 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00041810000000000003 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.813647+0000", 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00064208099999999997 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:28.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816722+0000", 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00307443 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.817497+0000", 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077593300000000003 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.817577+0000", 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9682e-05 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:28.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.817653+0000", 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5858000000000005e-05 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:28.989 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:28.990 INFO:teuthology.orchestra.run.smithi063.stdout:4373 read 0xe08e355 thru 0xe092822 (0x44ce bytes) 2024-09-13T09:27:28.990 INFO:teuthology.orchestra.run.smithi063.stdout:4374 write 0x4970747 thru 0x4977bbd (0x7477 bytes) 2024-09-13T09:27:28.991 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:28.992 INFO:teuthology.orchestra.run.smithi063.stdout:4375 write 0x6c00819 thru 0x6c06ebd (0x66a5 bytes) 2024-09-13T09:27:28.992 INFO:teuthology.orchestra.run.smithi063.stdout:4376 trunc from 0xec80a03 to 0x41ea570 2024-09-13T09:27:28.992 INFO:teuthology.orchestra.run.smithi063.stdout:4377 writesame 0xa9c81ee thru 0xa9cbe58 data_size 0x1(0x3c6b bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4378 read 0x189b652 thru 0x18a48e5 (0x9294 bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4379 punch from 0x35c81d3 to 0x35cdeff, (0x5d2c bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4380 writesame 0x6606aad thru 0x660fbc4 data_size 0x4(0x9118 bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4383 read 0x6c126db thru 0x6c2194e (0xf274 bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4384 read 0xa7bc3e7 thru 0xa7c99d5 (0xd5ef bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4386 write 0x1d7679b thru 0x1d76c78 (0x4de bytes) 2024-09-13T09:27:28.993 INFO:teuthology.orchestra.run.smithi063.stdout:4387 writesame 0x2610fb3 thru 0x2613f7d data_size 0x1(0x2fcb bytes) 2024-09-13T09:27:29.002 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:29.002 INFO:teuthology.orchestra.run.smithi063.stdout:4389 writesame 0x1629d08 thru 0x162e031 data_size 0x1(0x432a bytes) 2024-09-13T09:27:29.002 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:28.996+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:29.014 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:29.014 INFO:teuthology.orchestra.run.smithi063.stdout:4391 write 0xa589d thru 0xaaa78 (0x51dc bytes) 2024-09-13T09:27:29.028 INFO:teuthology.orchestra.run.smithi063.stdout:4393 writesame 0x8ff8f6a thru 0x9008f43 data_size 0x1(0xffda bytes) 2024-09-13T09:27:29.061 INFO:teuthology.orchestra.run.smithi063.stdout:4396 read 0x95a46e7 thru 0x95aa411 (0x5d2b bytes) 2024-09-13T09:27:29.061 INFO:teuthology.orchestra.run.smithi063.stdout:4397 write 0x1364734 thru 0x13666ed (0x1fba bytes) 2024-09-13T09:27:29.067 INFO:teuthology.orchestra.run.smithi063.stdout:4399 punch from 0x9775b81 to 0x97810e6, (0xb565 bytes) 2024-09-13T09:27:29.067 INFO:teuthology.orchestra.run.smithi063.stdout:4400 writesame 0x6363217 thru 0x636ceb7 data_size 0x1(0x9ca1 bytes) 2024-09-13T09:27:29.093 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:29.088+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:29.093 INFO:teuthology.orchestra.run.smithi063.stdout:4401 trunc from 0xa9cbe59 to 0x73615b7 2024-09-13T09:27:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:4402 read 0x47e730b thru 0x47ee8be (0x75b4 bytes) 2024-09-13T09:27:29.126 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:29.126 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:29.136 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:29.138 INFO:teuthology.orchestra.run.smithi063.stdout:4405 write 0xe067424 thru 0xe070ad7 (0x96b4 bytes) 2024-09-13T09:27:29.148 INFO:teuthology.orchestra.run.smithi063.stdout:4406 read 0xc6bf32c thru 0xc6c3717 (0x43ec bytes) 2024-09-13T09:27:29.148 INFO:teuthology.orchestra.run.smithi063.stdout:4409 write 0xaeabf9b thru 0xaeaf204 (0x326a bytes) 2024-09-13T09:27:29.153 INFO:teuthology.orchestra.run.smithi063.stdout:4410 write 0xc6245a5 thru 0xc6314b3 (0xcf0f bytes) 2024-09-13T09:27:29.165 INFO:teuthology.orchestra.run.smithi063.stdout:4411 writesame 0x7ae666d thru 0x7aeeae7 data_size 0x77(0x847b bytes) 2024-09-13T09:27:29.173 INFO:teuthology.orchestra.run.smithi063.stdout:4413 punch from 0x842dc55 to 0x843c969, (0xed14 bytes) 2024-09-13T09:27:29.173 INFO:teuthology.orchestra.run.smithi063.stdout:4415 writesame 0x8e0bd40 thru 0x8e0f3e7 data_size 0x35(0x36a8 bytes) 2024-09-13T09:27:29.179 INFO:teuthology.orchestra.run.smithi063.stdout:4417 writesame 0x2dec3df thru 0x2df4410 data_size 0x3d(0x8032 bytes) 2024-09-13T09:27:29.187 INFO:teuthology.orchestra.run.smithi063.stdout:4418 punch from 0x6e7e26 to 0x6f160d, (0x97e7 bytes) 2024-09-13T09:27:29.187 INFO:teuthology.orchestra.run.smithi063.stdout:4419 write 0xb83bca1 thru 0xb8437ea (0x7b4a bytes) 2024-09-13T09:27:29.194 INFO:teuthology.orchestra.run.smithi063.stdout:4420 read 0x401f4ca thru 0x402e167 (0xec9e bytes) 2024-09-13T09:27:29.194 INFO:teuthology.orchestra.run.smithi063.stdout:4421 punch from 0x7f75452 to 0x7f82c07, (0xd7b5 bytes) 2024-09-13T09:27:29.194 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:29.191+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:29.215 INFO:teuthology.orchestra.run.smithi063.stdout:4422 trunc from 0xe070ad8 to 0x1afa96b 2024-09-13T09:27:29.241 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:27:29.261 INFO:teuthology.orchestra.run.smithi063.stdout:4423 write 0x4f89780 thru 0x4f927d6 (0x9057 bytes) 2024-09-13T09:27:29.269 INFO:teuthology.orchestra.run.smithi063.stdout:4424 punch from 0xaf3f96 to 0xb031b7, (0xf221 bytes) 2024-09-13T09:27:29.269 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:29.269 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:29.282 INFO:teuthology.orchestra.run.smithi063.stdout:4425 trunc from 0x4f927d7 to 0xc7a330f 2024-09-13T09:27:29.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:29.310 INFO:teuthology.orchestra.run.smithi063.stdout:4426 write 0xe114289 thru 0xe115446 (0x11be bytes) 2024-09-13T09:27:29.311 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:29.304+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:29.318 INFO:teuthology.orchestra.run.smithi063.stdout:4428 punch from 0x30d41ff to 0x30e0159, (0xbf5a bytes) 2024-09-13T09:27:29.339 INFO:teuthology.orchestra.run.smithi063.stdout:4429 trunc from 0xe115447 to 0x3a55cd0 2024-09-13T09:27:29.364 INFO:teuthology.orchestra.run.smithi063.stdout:4430 read 0xe38eea thru 0xe40723 (0x783a bytes) 2024-09-13T09:27:29.370 INFO:teuthology.orchestra.run.smithi063.stdout:4431 trunc from 0x3a55cd0 to 0x1168cf3 2024-09-13T09:27:29.393 INFO:teuthology.orchestra.run.smithi063.stdout:4433 clone 34 order 23 su 32768 sc 8 2024-09-13T09:27:29.395 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:29.391+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:29.428 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.861478564999999, 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:29.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:29.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:29.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:29.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87135 3.1b 3:d87e56f6:::rbd_data.11181839c212.0000000000000020:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.633593515999998, 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085077469999999995, 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87135 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:29.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789975+0000", 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789989+0000", 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790012+0000", 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2889999999999999e-05 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790034+0000", 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2221e-05 2024-09-13T09:27:29.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790129+0000", 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5093000000000006e-05 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790376+0000", 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024744900000000001 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790600+0000", 2024-09-13T09:27:29.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223654 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.791728+0000", 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011281170000000001 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798262+0000", 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065342899999999999 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798385+0000", 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012293199999999999 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798486+0000", 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100324 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.441 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 39.262254472000002, 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.442 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:29.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.443 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:29.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:29.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:29.450 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 39.153909282999997, 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:29.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:29.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:29.457 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.615564329000001, 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.463 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.615405574, 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:29.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:29.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:29.470 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.615329633999998, 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:29.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.614350191, 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:29.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.606771471000002, 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:29.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:29.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:29.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.486 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.340025507, 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:29.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:29.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.391954475999999, 2024-09-13T09:27:29.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114301, 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031615+0000", 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031627+0000", 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2072999999999999e-05 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031658+0000", 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1037000000000001e-05 2024-09-13T09:27:29.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031679+0000", 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0557e-05 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040706+0000", 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090265759999999997 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040717+0000", 2024-09-13T09:27:29.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1357e-05 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040731+0000", 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4355000000000001e-05 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.497 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.391807604, 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114278, 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031763+0000", 2024-09-13T09:27:29.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031769+0000", 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4659999999999996e-06 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031799+0000", 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9164000000000001e-05 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031812+0000", 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3465999999999999e-05 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040746+0000", 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0089339659999999998 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040752+0000", 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7830000000000004e-06 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040878+0000", 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012623300000000001 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:27:29.502 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.273820186, 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:29.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:27:29.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:27:29.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:27:29.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:27:29.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128541 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.1572818729999996, 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085633649999999999, 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.508 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128541 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:29.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266289+0000", 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266293+0000", 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0759999999999996e-06 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266301+0000", 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4839999999999998e-06 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266306+0000", 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5860000000000004e-06 2024-09-13T09:27:29.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270348+0000", 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040418859999999997 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270447+0000", 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8554000000000005e-05 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270812+0000", 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000364932 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.273865+0000", 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030535940000000002 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274619+0000", 2024-09-13T09:27:29.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00075377899999999999 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274786+0000", 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000167298 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274853+0000", 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6674000000000006e-05 2024-09-13T09:27:29.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128959 3.1f 3:fd5f8e4a:::rbd_data.11181500dc3e.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.5878355969999998, 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018621092999999998, 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128959 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835734+0000", 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835742+0000", 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1089999999999998e-06 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835759+0000", 2024-09-13T09:27:29.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6591000000000001e-05 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835766+0000", 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835845+0000", 2024-09-13T09:27:29.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9524000000000006e-05 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835977+0000", 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013168399999999999 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.839738+0000", 2024-09-13T09:27:29.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003761398 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854287+0000", 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014548442999999999 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854295+0000", 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5690000000000005e-06 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854329+0000", 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4198e-05 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854357+0000", 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7491000000000002e-05 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.521 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129111 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.2969909120000001, 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086272250000000005, 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129111 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126580+0000", 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126586+0000", 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.178e-06 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126599+0000", 2024-09-13T09:27:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3124e-05 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126609+0000", 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0294e-05 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.130890+0000", 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042807009999999996 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131001+0000", 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110368 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131416+0000", 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00041545800000000001 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134852+0000", 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034356009999999999 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135120+0000", 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026872400000000002 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135173+0000", 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3149000000000002e-05 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135208+0000", 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4297e-05 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129312 3.1 3:875c0977:::rbd_data.11181500dc3e.00000000000000ea:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8840590339999999, 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086345329999999998, 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129312 2024-09-13T09:27:29.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539511+0000", 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539520+0000", 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0629999999999995e-06 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539535+0000", 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5242000000000001e-05 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539546+0000", 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1185999999999999e-05 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539603+0000", 2024-09-13T09:27:29.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7358000000000002e-05 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539688+0000", 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4495999999999999e-05 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.541358+0000", 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016698030000000001 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548023+0000", 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066649250000000004 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548069+0000", 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6715000000000003e-05 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548116+0000", 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6226000000000001e-05 2024-09-13T09:27:29.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548147+0000", 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1301999999999999e-05 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129371 3.1f 3:fe73dde2:::rbd_data.11181500dc3e.000000000000000f:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.781716141, 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014271700999999999, 2024-09-13T09:27:29.535 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129371 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641854+0000", 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641862+0000", 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641877+0000", 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5327000000000001e-05 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641886+0000", 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4129999999999995e-06 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641935+0000", 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8775e-05 2024-09-13T09:27:29.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.642014+0000", 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8574999999999999e-05 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:29.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.644683+0000", 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026697359999999998 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655832+0000", 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011148371000000001 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655974+0000", 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014196400000000001 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656079+0000", 2024-09-13T09:27:29.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010503800000000001 2024-09-13T09:27:29.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656127+0000", 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.812e-05 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130769 3.1 3:84b2050c:::rbd_data.1118de6ae5af.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.38846643800000002, 2024-09-13T09:27:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024401849, 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130769 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035103+0000", 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035114+0000", 2024-09-13T09:27:29.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1613e-05 2024-09-13T09:27:29.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035136+0000", 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1472e-05 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035145+0000", 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4860000000000003e-06 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035204+0000", 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9027000000000003e-05 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035363+0000", 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015868600000000001 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.040402+0000", 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050390000000000001 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059321+0000", 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018919062 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059340+0000", 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9001999999999999e-05 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059432+0000", 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1535999999999999e-05 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059507+0000", 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5220000000000001e-05 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130832 3.12 3:4988a482:::rbd_data.1118de6ae5af.000000000000005e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.204857861, 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090196749999999996, 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.550 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130832 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218710+0000", 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218726+0000", 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6439999999999998e-05 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218794+0000", 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7726999999999997e-05 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218815+0000", 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1449000000000001e-05 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218935+0000", 2024-09-13T09:27:29.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011945199999999999 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219123+0000", 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000187966 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219363+0000", 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024062399999999999 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.222351+0000", 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029879989999999999 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227642+0000", 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052903109999999998 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227693+0000", 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1786000000000002e-05 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227733+0000", 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9761000000000001e-05 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.556 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:29.558 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:29.558 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:29.552+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:29.658 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:29.655+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:29.720 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:29.720 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:29.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:29.760 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:29.755+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:29.838 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:29.838 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:29.838 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:29.838 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:29.838 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:29.848 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:29.861 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:29.856+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:29.960 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 80.001900105999994, 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:29.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:29.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:29.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.966 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 53.309487466, 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:27:29.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:27:29.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 53.309366904000001, 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:29.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.836593721, 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:29.975 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:29.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:29.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:29.981 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406240725000004, 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:29.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406214851000001, 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:29.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406189486999999, 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:29.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406104464999999, 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:29.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:29.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:29.999 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406071070000003, 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:30.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:30.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406057132000001, 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:30.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:30.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:30.008 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.406019385999997, 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:30.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:30.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:30.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.40600147, 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:30.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405987822, 2024-09-13T09:27:30.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:30.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:30.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405921177000003, 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:30.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:30.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405873116999999, 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:30.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.030 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405815844999999, 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:30.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:30.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405795736000002, 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.035 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:30.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405773590000003, 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:30.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405717322999998, 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:30.046 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.405701426999997, 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:30.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:30.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.051 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:30.051 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:30.051 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:30.052 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:30.053 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:30.048+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:30.154 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:30.148+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:30.211 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:30.212 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:30.212 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:30.212 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:30.222 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:30.254 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:30.248+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:30.278 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:30.355 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:30.348+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:30.361 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:30.361 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:30.361 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:30.361 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:30.361 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:30.376 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:30.388 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone33 intact 2024-09-13T09:27:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:30.456 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:30.448+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:30.462 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:30.478 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T09:27:30.481 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:30.490 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:30.490 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:30.503 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:30.504 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:30.525 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:30.534 INFO:teuthology.orchestra.run.smithi063.stdout:4434 trunc from 0x1168cf3 to 0xbf20631 2024-09-13T09:27:30.550 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:30.550 INFO:teuthology.orchestra.run.smithi063.stdout:4435 read 0xb687784 thru 0xb68e31f (0x6b9c bytes) 2024-09-13T09:27:30.550 INFO:teuthology.orchestra.run.smithi063.stdout:4436 punch from 0x97ba4a6 to 0x97c57d7, (0xb331 bytes) 2024-09-13T09:27:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:4439 writesame 0x4366c62 thru 0x4374b16 data_size 0x47(0xdeb5 bytes) 2024-09-13T09:27:30.557 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:30.552+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:30.564 INFO:teuthology.orchestra.run.smithi063.stdout:4441 trunc from 0xbf20631 to 0xa3d0f46 2024-09-13T09:27:30.573 INFO:teuthology.orchestra.run.smithi063.stdout:4442 writesame 0x5ec61b5 thru 0x5ed42ce data_size 0x1(0xe11a bytes) 2024-09-13T09:27:30.574 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:30.600 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:30.602 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:30.603 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:30.612 INFO:teuthology.orchestra.run.smithi063.stdout:4444 write 0x3b2ff0 thru 0x3b8810 (0x5821 bytes) 2024-09-13T09:27:30.623 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:30.624 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:30.633 INFO:teuthology.orchestra.run.smithi063.stdout:4445 write 0x3154290 thru 0x315a4eb (0x625c bytes) 2024-09-13T09:27:30.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:30.652 INFO:teuthology.orchestra.run.smithi063.stdout:4446 read 0x8f6d1ec thru 0x8f78de3 (0xbbf8 bytes) 2024-09-13T09:27:30.652 INFO:teuthology.orchestra.run.smithi063.stdout:4447 writesame 0x23a2a50 thru 0x23ac89a data_size 0x1(0x9e4b bytes) 2024-09-13T09:27:30.658 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:30.652+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:30.699 INFO:teuthology.orchestra.run.smithi063.stdout:4449 write 0xee5a2ce thru 0xee6645e (0xc191 bytes) 2024-09-13T09:27:30.707 INFO:teuthology.orchestra.run.smithi063.stdout:4451 writesame 0x62a8643 thru 0x62aa389 data_size 0x1(0x1d47 bytes) 2024-09-13T09:27:30.715 INFO:teuthology.orchestra.run.smithi063.stdout:4452 writesame 0x198b098 thru 0x1993297 data_size 0x34(0x8200 bytes) 2024-09-13T09:27:30.744 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:30.744 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:30.754 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:30.759 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:30.755+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:30.761 INFO:teuthology.orchestra.run.smithi063.stdout:4453 trunc from 0xee6645f to 0x1bc46b4 2024-09-13T09:27:30.800 INFO:teuthology.orchestra.run.smithi063.stdout:4454 punch from 0x7d6b42 to 0x7e6a5b, (0xff19 bytes) 2024-09-13T09:27:30.813 INFO:teuthology.orchestra.run.smithi063.stdout:4455 trunc from 0x1bc46b4 to 0x81be486 2024-09-13T09:27:30.824 INFO:teuthology.orchestra.run.smithi063.stdout:4457 read 0x1d6652d thru 0x1d720ae (0xbb82 bytes) 2024-09-13T09:27:30.825 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:30.850 INFO:teuthology.orchestra.run.smithi063.stdout:4460 writesame 0xe9f167c thru 0xe9fbaa4 data_size 0x5(0xa429 bytes) 2024-09-13T09:27:30.851 INFO:teuthology.orchestra.run.smithi063.stdout:4461 writesame 0x5fdec0e thru 0x5fe4ba0 data_size 0x1(0x5f93 bytes) 2024-09-13T09:27:30.859 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:30.852+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:30.860 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:30.860 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:30.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:30.874 INFO:teuthology.orchestra.run.smithi063.stdout:4463 read 0xbe29043 thru 0xbe2ea7a (0x5a38 bytes) 2024-09-13T09:27:30.874 INFO:teuthology.orchestra.run.smithi063.stdout:4464 writesame 0x10ed447 thru 0x10ee543 data_size 0x1(0x10fd bytes) 2024-09-13T09:27:30.878 INFO:teuthology.orchestra.run.smithi063.stdout:4467 write 0x5a034c8 thru 0x5a061c7 (0x2d00 bytes) 2024-09-13T09:27:30.885 INFO:teuthology.orchestra.run.smithi063.stdout:4468 punch from 0x9c56a43 to 0x9c5b46d, (0x4a2a bytes) 2024-09-13T09:27:30.885 INFO:teuthology.orchestra.run.smithi063.stdout:4469 write 0x8b924f4 thru 0x8b94d9d (0x28aa bytes) 2024-09-13T09:27:30.891 INFO:teuthology.orchestra.run.smithi063.stdout:4470 read 0x1928862 thru 0x1937722 (0xeec1 bytes) 2024-09-13T09:27:30.892 INFO:teuthology.orchestra.run.smithi063.stdout:4471 read 0x3957349 thru 0x3959593 (0x224b bytes) 2024-09-13T09:27:30.893 INFO:teuthology.orchestra.run.smithi063.stdout:4472 writesame 0x6cc170e thru 0x6cc1dc3 data_size 0x1(0x6b6 bytes) 2024-09-13T09:27:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:4473 punch from 0xbe0a1fa to 0xbe102bb, (0x60c1 bytes) 2024-09-13T09:27:30.897 INFO:teuthology.orchestra.run.smithi063.stdout:4474 write 0xd73004 thru 0xd771ac (0x41a9 bytes) 2024-09-13T09:27:30.919 INFO:teuthology.orchestra.run.smithi063.stdout:4475 trunc from 0xe9fbaa5 to 0x5ba7865 2024-09-13T09:27:30.943 INFO:teuthology.orchestra.run.smithi063.stdout:4477 trunc from 0x5ba7865 to 0x706c1bb 2024-09-13T09:27:30.957 INFO:teuthology.orchestra.run.smithi063.stdout:4478 read 0x114701f thru 0x114d88b (0x686d bytes) 2024-09-13T09:27:30.958 INFO:teuthology.orchestra.run.smithi063.stdout:4479 punch from 0x23edb91 to 0x23fd0ae, (0xf51d bytes) 2024-09-13T09:27:30.958 INFO:teuthology.orchestra.run.smithi063.stdout:4481 read 0xe32aef thru 0xe3d228 (0xa73a bytes) 2024-09-13T09:27:30.960 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:30.952+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:30.971 INFO:teuthology.orchestra.run.smithi063.stdout:4482 trunc from 0x706c1bb to 0x1764a4f 2024-09-13T09:27:30.980 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:30.980 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:30.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:30.995 INFO:teuthology.orchestra.run.smithi063.stdout:4483 read 0x6ffb1 thru 0x72bee (0x2c3e bytes) 2024-09-13T09:27:31.011 INFO:teuthology.orchestra.run.smithi063.stdout:4484 trunc from 0x1764a4f to 0x8447d13 2024-09-13T09:27:31.023 INFO:teuthology.orchestra.run.smithi063.stdout:4485 punch from 0x4f0e99d to 0x4f1033c, (0x199f bytes) 2024-09-13T09:27:31.023 INFO:teuthology.orchestra.run.smithi063.stdout:4486 read 0x54b5abf thru 0x54ba48b (0x49cd bytes) 2024-09-13T09:27:31.023 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.023 INFO:teuthology.orchestra.run.smithi063.stdout:4489 write 0x97ec36 thru 0x9818a0 (0x2c6b bytes) 2024-09-13T09:27:31.026 INFO:teuthology.orchestra.run.smithi063.stdout:4493 read 0x2655dce thru 0x266204e (0xc281 bytes) 2024-09-13T09:27:31.027 INFO:teuthology.orchestra.run.smithi063.stdout:4494 read 0x14c1d43 thru 0x14c5b2a (0x3de8 bytes) 2024-09-13T09:27:31.028 INFO:teuthology.orchestra.run.smithi063.stdout:4495 read 0x20202de thru 0x2027152 (0x6e75 bytes) 2024-09-13T09:27:31.029 INFO:teuthology.orchestra.run.smithi063.stdout:4496 read 0x4f508f0 thru 0x4f5906b (0x877c bytes) 2024-09-13T09:27:31.053 INFO:teuthology.orchestra.run.smithi063.stdout:4497 write 0xd987837 thru 0xd98bbf8 (0x43c2 bytes) 2024-09-13T09:27:31.061 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:31.055+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:31.061 INFO:teuthology.orchestra.run.smithi063.stdout:4498 write 0xcde3d7b thru 0xcdf13a7 (0xd62d bytes) 2024-09-13T09:27:31.070 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.070 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.070 INFO:teuthology.orchestra.run.smithi063.stdout:4503 write 0x7f3ecc thru 0x7f9e8b (0x5fc0 bytes) 2024-09-13T09:27:31.073 INFO:teuthology.orchestra.run.smithi063.stdout:4506 writesame 0x3577588 thru 0x357c827 data_size 0x10(0x52a0 bytes) 2024-09-13T09:27:31.076 INFO:teuthology.orchestra.run.smithi063.stdout:4507 write 0x463706 thru 0x464dc4 (0x16bf bytes) 2024-09-13T09:27:31.079 INFO:teuthology.orchestra.run.smithi063.stdout:4508 write 0x6839d5f thru 0x684836e (0xe610 bytes) 2024-09-13T09:27:31.088 INFO:teuthology.orchestra.run.smithi063.stdout:4509 punch from 0x9399dde to 0x93a2b93, (0x8db5 bytes) 2024-09-13T09:27:31.088 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.088 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.088 INFO:teuthology.orchestra.run.smithi063.stdout:4513 write 0xb7f5f35 thru 0xb80565a (0xf726 bytes) 2024-09-13T09:27:31.093 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.093 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.104 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:31.109 INFO:teuthology.orchestra.run.smithi063.stdout:4514 trunc from 0xd98bbf9 to 0x781fd29 2024-09-13T09:27:31.132 INFO:teuthology.orchestra.run.smithi063.stdout:4515 trunc from 0x781fd29 to 0x92cdcd7 2024-09-13T09:27:31.141 INFO:teuthology.orchestra.run.smithi063.stdout:4516 punch from 0x895383f to 0x8954b82, (0x1343 bytes) 2024-09-13T09:27:31.159 INFO:teuthology.orchestra.run.smithi063.stdout:4517 trunc from 0x92cdcd7 to 0x69b7fa 2024-09-13T09:27:31.162 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:31.156+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:31.203 INFO:teuthology.orchestra.run.smithi063.stdout:4518 write 0x35946ad thru 0x3598444 (0x3d98 bytes) 2024-09-13T09:27:31.208 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.208 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:31.223 INFO:teuthology.orchestra.run.smithi063.stdout:4519 writesame 0x55a6afa thru 0x55adb93 data_size 0x1d(0x709a bytes) 2024-09-13T09:27:31.223 INFO:teuthology.orchestra.run.smithi063.stdout:4520 write 0xdcba85 thru 0xdcdbbe (0x213a bytes) 2024-09-13T09:27:31.239 INFO:teuthology.orchestra.run.smithi063.stdout:4521 write 0x62b62b9 thru 0x62bd799 (0x74e1 bytes) 2024-09-13T09:27:31.253 INFO:teuthology.orchestra.run.smithi063.stdout:4523 trunc from 0x62bd79a to 0x36326b3 2024-09-13T09:27:31.257 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:27:31.262 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:31.259+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:31.280 INFO:teuthology.orchestra.run.smithi063.stdout:4524 punch from 0x34de90c to 0x34e7aab, (0x919f bytes) 2024-09-13T09:27:31.280 INFO:teuthology.orchestra.run.smithi063.stdout:4525 read 0x25e793d thru 0x25ea6e9 (0x2dad bytes) 2024-09-13T09:27:31.280 INFO:teuthology.orchestra.run.smithi063.stdout:4527 punch from 0x1cf66b4 to 0x1d0228b, (0xbbd7 bytes) 2024-09-13T09:27:31.280 INFO:teuthology.orchestra.run.smithi063.stdout:4528 read 0x24c5a9e thru 0x24cc09e (0x6601 bytes) 2024-09-13T09:27:31.310 INFO:teuthology.orchestra.run.smithi063.stdout:4529 write 0xc124ab0 thru 0xc12a6ec (0x5c3d bytes) 2024-09-13T09:27:31.319 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.319 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.320 INFO:teuthology.orchestra.run.smithi063.stdout:4530 read 0x33dd074 thru 0x33ec091 (0xf01e bytes) 2024-09-13T09:27:31.321 INFO:teuthology.orchestra.run.smithi063.stdout:4531 read 0xaffb7a9 thru 0xb000db3 (0x560b bytes) 2024-09-13T09:27:31.321 INFO:teuthology.orchestra.run.smithi063.stdout:4532 read 0x302c1ec thru 0x303a6d1 (0xe4e6 bytes) 2024-09-13T09:27:31.322 INFO:teuthology.orchestra.run.smithi063.stdout:4533 punch from 0x7f9eaf4 to 0x7fa4fbe, (0x64ca bytes) 2024-09-13T09:27:31.324 INFO:teuthology.orchestra.run.smithi063.stdout:4534 trunc from 0xc12a6ed to 0xd303701 2024-09-13T09:27:31.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:31.334 INFO:teuthology.orchestra.run.smithi063.stdout:4535 write 0x32aa9db thru 0x32b9c16 (0xf23c bytes) 2024-09-13T09:27:31.338 INFO:teuthology.orchestra.run.smithi063.stdout:4537 read 0x810ac6a thru 0x811a4ff (0xf896 bytes) 2024-09-13T09:27:31.338 INFO:teuthology.orchestra.run.smithi063.stdout:4538 writesame 0x8b8ae01 thru 0x8b97378 data_size 0x8e(0xc578 bytes) 2024-09-13T09:27:31.346 INFO:teuthology.orchestra.run.smithi063.stdout:4540 punch from 0x196b14e to 0x1970470, (0x5322 bytes) 2024-09-13T09:27:31.356 INFO:teuthology.orchestra.run.smithi063.stdout:4542 trunc from 0xd303701 to 0x848adb2 2024-09-13T09:27:31.374 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:31.371+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:31.391 INFO:teuthology.orchestra.run.smithi063.stdout:4543 write 0xd5b2e15 thru 0xd5c1426 (0xe612 bytes) 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:4546 write 0x93f0733 thru 0x93fcd80 (0xc64e bytes) 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:4548 write 0x212a4d0 thru 0x2138bbe (0xe6ef bytes) 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:4549 write 0xa7a9c82 thru 0xa7b7f50 (0xe2cf bytes) 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:4552 punch from 0x70c0f9a to 0x70c79d6, (0x6a3c bytes) 2024-09-13T09:27:31.483 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4554 write 0x4c6d3d4 thru 0x4c70ebd (0x3aea bytes) 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4555 write 0x241e9d4 thru 0x242b3fa (0xca27 bytes) 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4556 punch from 0xa091a14 to 0xa09ce14, (0xb400 bytes) 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4557 trunc from 0xd5c1427 to 0x3b23f79 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4560 read 0x385ccd9 thru 0x386b494 (0xe7bc bytes) 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4562 punch from 0x1711d24 to 0x1718f2e, (0x720a bytes) 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4564 read 0x25b0485 thru 0x25b1030 (0xbac bytes) 2024-09-13T09:27:31.484 INFO:teuthology.orchestra.run.smithi063.stdout:4565 read 0x8faff0 thru 0x909514 (0xe525 bytes) 2024-09-13T09:27:31.485 INFO:teuthology.orchestra.run.smithi063.stdout:4567 write 0x22db74a thru 0x22e4b6c (0x9423 bytes) 2024-09-13T09:27:31.485 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:31.479+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:31.486 INFO:teuthology.orchestra.run.smithi063.stdout:4570 write 0x89e2cd4 thru 0x89ec042 (0x936f bytes) 2024-09-13T09:27:31.489 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.489 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.492 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:31.492 INFO:teuthology.orchestra.run.smithi063.stdout:4572 write 0xe06a31 thru 0xe11bd2 (0xb1a2 bytes) 2024-09-13T09:27:31.496 INFO:teuthology.orchestra.run.smithi063.stdout:4573 punch from 0x2feb6ae to 0x2ff3aab, (0x83fd bytes) 2024-09-13T09:27:31.499 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:31.508 INFO:teuthology.orchestra.run.smithi063.stdout:4575 write 0xb0f0e8e thru 0xb0f21d9 (0x134c bytes) 2024-09-13T09:27:31.514 INFO:teuthology.orchestra.run.smithi063.stdout:4576 punch from 0x4fd41b0 to 0x4fd7d4f, (0x3b9f bytes) 2024-09-13T09:27:31.514 INFO:teuthology.orchestra.run.smithi063.stdout:4577 read 0xabb0212 thru 0xabb99cb (0x97ba bytes) 2024-09-13T09:27:31.514 INFO:teuthology.orchestra.run.smithi063.stdout:4579 clone 35 order 20 su 262144 sc 8 2024-09-13T09:27:31.582 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:31.579+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:31.615 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.615 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:31.682 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:31.679+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:31.783 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:31.779+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:31.786 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.786 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:31.884 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:31.879+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:31.948 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:31.948 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:31.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:32.002 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:31.999+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:32.065 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:32.065 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:32.075 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:32.086 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:32.083+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:32.187 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:32.180+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:32.191 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:32.191 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:32.202 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:32.243 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:32.243 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:32.244 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-13T09:27:32.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-13T09:27:32.266 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:27:32.288 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:32.284+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:32.344 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:32.344 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:32.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:32.387 INFO:teuthology.orchestra.run.smithi063.stdout:leaving image image_client.0-clone34 intact 2024-09-13T09:27:32.389 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:32.387+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:32.490 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:32.487+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:32.501 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:32.501 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:32.512 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:32.586 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:27:32.580+0000 7f00df3a9700 -1 mon.a@0(leader).osd e101 definitely_dead 0 2024-09-13T09:27:32.590 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:32.587+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:32.607 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T09:27:32.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:32.618 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:32.618 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:32.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:32.691 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:32.684+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:32.696 INFO:teuthology.orchestra.run.smithi063.stdout:4580 punch from 0x3abac7e to 0x3ac6f25, (0xc2a7 bytes) 2024-09-13T09:27:32.703 INFO:teuthology.orchestra.run.smithi063.stdout:4581 writesame 0x1154deb thru 0x115a62a data_size 0x20(0x5840 bytes) 2024-09-13T09:27:32.714 INFO:teuthology.orchestra.run.smithi063.stdout:4583 punch from 0x931673 to 0x93452d, (0x2eba bytes) 2024-09-13T09:27:32.714 INFO:teuthology.orchestra.run.smithi063.stdout:4585 punch from 0x7ee35a2 to 0x7eeae25, (0x7883 bytes) 2024-09-13T09:27:32.727 INFO:teuthology.orchestra.run.smithi063.stdout:4586 write 0xd31a82f thru 0xd3231ce (0x89a0 bytes) 2024-09-13T09:27:32.747 INFO:teuthology.orchestra.run.smithi063.stdout:4587 trunc from 0xd3231cf to 0x53c0507 2024-09-13T09:27:32.766 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:32.782 INFO:teuthology.orchestra.run.smithi063.stdout:4588 read 0xf95040 thru 0xf9e842 (0x9803 bytes) 2024-09-13T09:27:32.791 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:32.787+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:32.804 INFO:teuthology.orchestra.run.smithi063.stdout:4591 writesame 0xc800380 thru 0xc80a645 data_size 0x2d(0xa2c6 bytes) 2024-09-13T09:27:32.804 INFO:teuthology.orchestra.run.smithi063.stdout:4592 write 0x4e2eda6 thru 0x4e3105c (0x22b7 bytes) 2024-09-13T09:27:32.804 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:32.812 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:32.812 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:32.812 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:32.812 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:32.812 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:32.813 INFO:teuthology.orchestra.run.smithi063.stdout:4593 trunc from 0xc80a646 to 0xb4ec406 2024-09-13T09:27:32.822 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:32.825 INFO:teuthology.orchestra.run.smithi063.stdout:4594 trunc from 0xb4ec406 to 0xa07ab41 2024-09-13T09:27:32.830 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:32.835 INFO:teuthology.orchestra.run.smithi063.stdout:4595 read 0x8a822b4 thru 0x8a8e46b (0xc1b8 bytes) 2024-09-13T09:27:32.835 INFO:teuthology.orchestra.run.smithi063.stdout:4596 writesame 0x3d5979 thru 0x3e268d data_size 0x1(0xcd15 bytes) 2024-09-13T09:27:32.855 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:32.887 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:32.892 INFO:teuthology.orchestra.run.smithi063.stdout:4597 write 0x93a3336 thru 0x93aa2e2 (0x6fad bytes) 2024-09-13T09:27:32.892 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:32.887+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:32.899 INFO:teuthology.orchestra.run.smithi063.stdout:4598 read 0x8834a10 thru 0x883c431 (0x7a22 bytes) 2024-09-13T09:27:32.907 INFO:teuthology.orchestra.run.smithi063.stdout:4600 trunc from 0xa07ab41 to 0x5fe7ddd 2024-09-13T09:27:32.910 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:32.926 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:32.926 INFO:teuthology.orchestra.run.smithi063.stdout:4603 read 0x2b2b841 thru 0x2b2bca2 (0x462 bytes) 2024-09-13T09:27:32.930 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:32.930 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:32.937 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:32.938 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.370924125000002, 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:32.939 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:32.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:32.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:87135 3.1b 3:d87e56f6:::rbd_data.11181839c212.0000000000000020:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 46.143039076000001, 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085077469999999995, 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 87135 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789978+0000", 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789975+0000", 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.789989+0000", 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790012+0000", 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2889999999999999e-05 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790034+0000", 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2221e-05 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790129+0000", 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5093000000000006e-05 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790376+0000", 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024744900000000001 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.790600+0000", 2024-09-13T09:27:32.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000223654 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.791728+0000", 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011281170000000001 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798262+0000", 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065342899999999999 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798385+0000", 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012293199999999999 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.798486+0000", 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100324 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.953 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.771700031999998, 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:32.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.663354843, 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.960 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:32.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.125009888999998, 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.966 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:32.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:32.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.124851133999996, 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:32.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:32.977 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.124775194000001, 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:32.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:32.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.983 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.123795751000003, 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:32.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:32.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:32.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:32.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:32.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 41.116217030999998, 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.990 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:32.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:32.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:32.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:32.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.849471067000003, 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:32.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:32.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:32.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:33.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.901400035999998, 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114301, 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031615+0000", 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031627+0000", 2024-09-13T09:27:33.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2072999999999999e-05 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031658+0000", 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1037000000000001e-05 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031679+0000", 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0557e-05 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040706+0000", 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090265759999999997 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040717+0000", 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1357e-05 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040731+0000", 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4355000000000001e-05 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.005 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.901253164, 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114278, 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:33.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031763+0000", 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031769+0000", 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4659999999999996e-06 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031799+0000", 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9164000000000001e-05 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031812+0000", 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3465999999999999e-05 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040746+0000", 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0089339659999999998 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040752+0000", 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7830000000000004e-06 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040878+0000", 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012623300000000001 2024-09-13T09:27:33.009 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.783265746000001, 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.010 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:27:33.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.014 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128541 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6667274330000001, 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085633649999999999, 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128541 2024-09-13T09:27:33.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266289+0000", 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266293+0000", 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0759999999999996e-06 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266301+0000", 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4839999999999998e-06 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266306+0000", 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5860000000000004e-06 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270348+0000", 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040418859999999997 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270447+0000", 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8554000000000005e-05 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270812+0000", 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000364932 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.273865+0000", 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030535940000000002 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274619+0000", 2024-09-13T09:27:33.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00075377899999999999 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274786+0000", 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000167298 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274853+0000", 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6674000000000006e-05 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128959 3.1f 3:fd5f8e4a:::rbd_data.11181500dc3e.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.0972811569999994, 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018621092999999998, 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128959 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835734+0000", 2024-09-13T09:27:33.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835742+0000", 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1089999999999998e-06 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835759+0000", 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6591000000000001e-05 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835766+0000", 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835845+0000", 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9524000000000006e-05 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835977+0000", 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013168399999999999 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.839738+0000", 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003761398 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854287+0000", 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014548442999999999 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854295+0000", 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5690000000000005e-06 2024-09-13T09:27:33.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854329+0000", 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4198e-05 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854357+0000", 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7491000000000002e-05 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129111 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.8064364719999997, 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086272250000000005, 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129111 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126580+0000", 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126586+0000", 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.178e-06 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126599+0000", 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3124e-05 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126609+0000", 2024-09-13T09:27:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0294e-05 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.130890+0000", 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042807009999999996 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131001+0000", 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110368 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131416+0000", 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00041545800000000001 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134852+0000", 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034356009999999999 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135120+0000", 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026872400000000002 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135173+0000", 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3149000000000002e-05 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135208+0000", 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4297e-05 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.032 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129312 3.1 3:875c0977:::rbd_data.11181500dc3e.00000000000000ea:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.3935045940000004, 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086345329999999998, 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129312 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539511+0000", 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539520+0000", 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0629999999999995e-06 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539535+0000", 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5242000000000001e-05 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539546+0000", 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1185999999999999e-05 2024-09-13T09:27:33.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539603+0000", 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7358000000000002e-05 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539688+0000", 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4495999999999999e-05 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.541358+0000", 2024-09-13T09:27:33.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016698030000000001 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548023+0000", 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066649250000000004 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548069+0000", 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6715000000000003e-05 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548116+0000", 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6226000000000001e-05 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548147+0000", 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1301999999999999e-05 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129371 3.1f 3:fe73dde2:::rbd_data.11181500dc3e.000000000000000f:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:33.038 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.2911617010000001, 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014271700999999999, 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129371 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641854+0000", 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641862+0000", 2024-09-13T09:27:33.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641877+0000", 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5327000000000001e-05 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641886+0000", 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4129999999999995e-06 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641935+0000", 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8775e-05 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.642014+0000", 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8574999999999999e-05 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.644683+0000", 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026697359999999998 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655832+0000", 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011148371000000001 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655974+0000", 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014196400000000001 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656079+0000", 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010503800000000001 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656127+0000", 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.812e-05 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130769 3.1 3:84b2050c:::rbd_data.1118de6ae5af.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.8979119980000001, 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024401849, 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130769 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035103+0000", 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035114+0000", 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1613e-05 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035136+0000", 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1472e-05 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035145+0000", 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4860000000000003e-06 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035204+0000", 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9027000000000003e-05 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035363+0000", 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015868600000000001 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.040402+0000", 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050390000000000001 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059321+0000", 2024-09-13T09:27:33.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018919062 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059340+0000", 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9001999999999999e-05 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059432+0000", 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1535999999999999e-05 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059507+0000", 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5220000000000001e-05 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130832 3.12 3:4988a482:::rbd_data.1118de6ae5af.000000000000005e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.7143034209999999, 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090196749999999996, 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130832 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218710+0000", 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218726+0000", 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6439999999999998e-05 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218794+0000", 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7726999999999997e-05 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218815+0000", 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1449000000000001e-05 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218935+0000", 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011945199999999999 2024-09-13T09:27:33.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219123+0000", 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000187966 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219363+0000", 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024062399999999999 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.222351+0000", 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029879989999999999 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227642+0000", 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052903109999999998 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227693+0000", 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1786000000000002e-05 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227733+0000", 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9761000000000001e-05 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.056 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:33.057 INFO:teuthology.orchestra.run.smithi063.stdout:4606 writesame 0x97ca1ac thru 0x97ccc0d data_size 0x32(0x2a62 bytes) 2024-09-13T09:27:33.057 INFO:teuthology.orchestra.run.smithi063.stdout:4607 read 0x4c53106 thru 0x4c5fa66 (0xc961 bytes) 2024-09-13T09:27:33.058 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:33.059 INFO:teuthology.orchestra.run.smithi063.stdout:4608 write 0x96bdca7 thru 0x96c9ad6 (0xbe30 bytes) 2024-09-13T09:27:33.059 INFO:teuthology.orchestra.run.smithi063.stdout:4609 read 0xc09ddd thru 0xc123b1 (0x85d5 bytes) 2024-09-13T09:27:33.059 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:33.060 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:33.061 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:33.061 INFO:teuthology.orchestra.run.smithi063.stdout:4610 writesame 0xcb5d2e3 thru 0xcb678dc data_size 0x23(0xa5fa bytes) 2024-09-13T09:27:33.061 INFO:teuthology.orchestra.run.smithi063.stdout:4611 punch from 0x1f2fb8b to 0x1f3e8ef, (0xed64 bytes) 2024-09-13T09:27:33.061 INFO:teuthology.orchestra.run.smithi063.stdout:4613 trunc from 0xcb678dd to 0xb37a2c 2024-09-13T09:27:33.062 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:33.052+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:33.062 INFO:teuthology.orchestra.run.smithi063.stdout:4615 trunc from 0xb37a2c to 0x4bdc8bb 2024-09-13T09:27:33.067 INFO:teuthology.orchestra.run.smithi063.stdout:4616 punch from 0x4baeb87 to 0x4bbaf8b, (0xc404 bytes) 2024-09-13T09:27:33.075 INFO:teuthology.orchestra.run.smithi063.stdout:4617 trunc from 0x4bdc8bb to 0xbee3c2 2024-09-13T09:27:33.127 INFO:teuthology.orchestra.run.smithi063.stdout:4620 writesame 0xe23ccfc thru 0xe23ec49 data_size 0x1(0x1f4e bytes) 2024-09-13T09:27:33.127 INFO:teuthology.orchestra.run.smithi063.stdout:4621 write 0xb4aae49 thru 0xb4b73cc (0xc584 bytes) 2024-09-13T09:27:33.134 INFO:teuthology.orchestra.run.smithi063.stdout:4622 writesame 0x1eb14f7 thru 0x1ebd714 data_size 0x1(0xc21e bytes) 2024-09-13T09:27:33.158 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:33.155+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:33.172 INFO:teuthology.orchestra.run.smithi063.stdout:4623 write 0x7f20697 thru 0x7f25819 (0x5183 bytes) 2024-09-13T09:27:33.178 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:33.178 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:33.178 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:33.178 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:33.180 INFO:teuthology.orchestra.run.smithi063.stdout:4624 write 0xb3b73b6 thru 0xb3b8126 (0xd71 bytes) 2024-09-13T09:27:33.186 INFO:teuthology.orchestra.run.smithi063.stdout:4625 write 0x8b9e012 thru 0x8ba0352 (0x2341 bytes) 2024-09-13T09:27:33.189 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:33.192 INFO:teuthology.orchestra.run.smithi063.stdout:4626 write 0x5fad5d6 thru 0x5fba064 (0xca8f bytes) 2024-09-13T09:27:33.211 INFO:teuthology.orchestra.run.smithi063.stdout:4628 trunc from 0xe23ec4a to 0x8562602 2024-09-13T09:27:33.231 INFO:teuthology.orchestra.run.smithi063.stdout:4629 read 0x182262f thru 0x182f44a (0xce1c bytes) 2024-09-13T09:27:33.231 INFO:teuthology.orchestra.run.smithi063.stdout:4630 punch from 0x4dcfbb to 0x4ddd51, (0xd96 bytes) 2024-09-13T09:27:33.231 INFO:teuthology.orchestra.run.smithi063.stdout:4632 read 0x1c63c2c thru 0x1c6979e (0x5b73 bytes) 2024-09-13T09:27:33.231 INFO:teuthology.orchestra.run.smithi063.stdout:4633 read 0x97c071 thru 0x97f51b (0x34ab bytes) 2024-09-13T09:27:33.232 INFO:teuthology.orchestra.run.smithi063.stdout:4635 read 0x57dd4ac thru 0x57e53dd (0x7f32 bytes) 2024-09-13T09:27:33.232 INFO:teuthology.orchestra.run.smithi063.stdout:4636 read 0x5aa38ca thru 0x5aa6fbe (0x36f5 bytes) 2024-09-13T09:27:33.232 INFO:teuthology.orchestra.run.smithi063.stdout:4639 read 0x5b67d3e thru 0x5b70183 (0x8446 bytes) 2024-09-13T09:27:33.232 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:33.232 INFO:teuthology.orchestra.run.smithi063.stdout:4642 write 0x25d9373 thru 0x25e5659 (0xc2e7 bytes) 2024-09-13T09:27:33.239 INFO:teuthology.orchestra.run.smithi063.stdout:4644 read 0x74970ab thru 0x74a0615 (0x956b bytes) 2024-09-13T09:27:33.239 INFO:teuthology.orchestra.run.smithi063.stdout:4645 write 0x408bf75 thru 0x408e0da (0x2166 bytes) 2024-09-13T09:27:33.259 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:33.255+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:33.265 INFO:teuthology.orchestra.run.smithi063.stdout:4646 writesame 0xb671b27 thru 0xb68185f data_size 0x5(0xfd39 bytes) 2024-09-13T09:27:33.265 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:33.275 INFO:teuthology.orchestra.run.smithi063.stdout:4649 trunc from 0xb681860 to 0x48a1192 2024-09-13T09:27:33.286 INFO:teuthology.orchestra.run.smithi063.stderr:marked in osd.4. 2024-09-13T09:27:33.297 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-13T09:27:33.304 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:33.304 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:33.304 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:33.305 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:33.305 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:33.314 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:33.360 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:33.355+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:33.461 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:33.459+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.511973515999998, 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.470 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:33.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:33.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:33.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38917 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:33.479 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.819560875999997, 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030745616, 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38917 2024-09-13T09:27:33.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646242+0000", 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646265+0000", 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3824000000000001e-05 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646279+0000", 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:27:33.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646301+0000", 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1696e-05 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676978+0000", 2024-09-13T09:27:33.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030677598 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.676988+0000", 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:33.484 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.819440313999998, 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:33.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:33.488 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.346667130999997, 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:33.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:33.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916314135, 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:33.494 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:33.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.498 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916288260999998, 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.499 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:33.503 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916262897000003, 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.504 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:33.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:33.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916177875000002, 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.509 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:33.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:33.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.514 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.91614448, 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:33.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916130541999998, 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:33.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:33.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.524 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916092796000001, 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:33.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916074879999996, 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.529 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.533 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916061231999997, 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:33.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.537 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915994587, 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:33.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:33.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915946527000003, 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:33.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:33.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:33.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915889255000003, 2024-09-13T09:27:33.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:33.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915869145999999, 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:33.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.557 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915846999999999, 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:33.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:33.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915790733000001, 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.562 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:33.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.915774837000001, 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:33.566 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.570 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:33.572 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:33.572 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:33.564+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:33.673 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:33.667+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:33.688 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:33.688 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:33.688 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:33.688 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:33.699 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:33.773 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:33.771+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:33.810 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:33.811 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:33.820 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:33.874 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:33.868+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:33.931 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:33.931 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:33.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:33.931 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:385 3.0 e27/21 3:0220607e:::rbd_header.111867511a81:head v 27'15, mlcod=27'13)", 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 94.124069153999997, 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035077139999999999, 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802939+0000", 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802937+0000", 2024-09-13T09:27:33.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802950+0000", 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2981e-05 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802972+0000", 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1991999999999999e-05 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.802980+0000", 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1839999999999999e-06 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803082+0000", 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000101953 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.803131+0000", 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9316e-05 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806372+0000", 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003240939 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:25:59.806446+0000", 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4047000000000004e-05 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:2062 3.9 e30/21 3:904f7f74:::rbd_data.1118e3e5e25d.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 90.715703145999996, 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0038927609999999998, 2024-09-13T09:27:33.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211305+0000", 2024-09-13T09:27:33.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211304+0000", 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211313+0000", 2024-09-13T09:27:33.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3079999999999997e-06 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211326+0000", 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3308e-05 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211335+0000", 2024-09-13T09:27:33.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9060000000000008e-06 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211400+0000", 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4831000000000003e-05 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.211430+0000", 2024-09-13T09:27:33.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0528000000000003e-05 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215139+0000", 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037088389999999998 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:03.215197+0000", 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8013999999999997e-05 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 88.525984088000001, 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.943 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:27:33.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:33.947 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.960874703000002, 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:33.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:33.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:33.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.88733569, 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:33.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:33.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.766731746000005, 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:33.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.962 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.752473963, 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:33.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:33.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:35686 3.4 e52/21 3:24a0389b:::rbd_data.11185dd0f6b.0000000000000002:head v 52'208, mlcod=51'205)", 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.283755509999999, 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040557910000000004, 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.967 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643252+0000", 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643250+0000", 2024-09-13T09:27:33.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643266+0000", 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5568000000000001e-05 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643296+0000", 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0318e-05 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643312+0000", 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5744999999999999e-05 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643405+0000", 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2878000000000001e-05 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.643437+0000", 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1841999999999998e-05 2024-09-13T09:27:33.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647219+0000", 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037825929999999999 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:31.647308+0000", 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8771999999999996e-05 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.294830816999998, 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:33.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.089804768999997, 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:33.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:33.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:33.980 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44985 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'110, mlcod=55'108)", 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.991466721000002, 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039613130000000002, 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935541+0000", 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935540+0000", 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935553+0000", 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.344e-05 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935583+0000", 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9944e-05 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935592+0000", 2024-09-13T09:27:33.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7620000000000001e-06 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935656+0000", 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4047000000000005e-05 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.935689+0000", 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3232000000000003e-05 2024-09-13T09:27:33.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939452+0000", 2024-09-13T09:27:33.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003763288 2024-09-13T09:27:33.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.939502+0000", 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9867999999999998e-05 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.916065545999999, 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:33.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:33.990 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.721983102999999, 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:33.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:33.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:79345 3.9 e62/21 3:911e51ed:::rbd_data.11181839c212.00000000000001b0:head v 62'122, mlcod=62'119)", 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.606361604, 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037627889999999999, 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:33.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320646+0000", 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320645+0000", 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320669+0000", 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4389999999999999e-05 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320687+0000", 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7411999999999999e-05 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320697+0000", 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.084e-05 2024-09-13T09:27:33.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320841+0000", 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000143647 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.320923+0000", 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1507000000000004e-05 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324293+0000", 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033704429999999999 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.324409+0000", 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000115858 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:33.999 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 46.337620262000002, 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:34.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:34.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:34.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128800 3.9 3:959ec624:::rbd_data.11181500dc3e.0000000000000007:head [stat,write 32768~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.622511+0000", 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3044962269999996, 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0035983650000000001, 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:34.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128800 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622511+0000", 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622511+0000", 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622508+0000", 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622581+0000", 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2686000000000002e-05 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622627+0000", 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5609000000000003e-05 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622648+0000", 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1248000000000001e-05 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622766+0000", 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011749800000000001 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.622937+0000", 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017193800000000001 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.623175+0000", 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023796900000000001 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.624812+0000", 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016366029999999999 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626029+0000", 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001217311 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626075+0000", 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5430999999999999e-05 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.626110+0000", 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5055000000000002e-05 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:128958 3.13 e95/94 3:ca28514f:::rbd_data.11181500dc3e.000000000000000a:head v 95'531, mlcod=95'529)", 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.079037864, 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042980650000000002, 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:34.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847968+0000", 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:34.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850296+0000", 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002328044 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850322+0000", 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5741e-05 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850334+0000", 2024-09-13T09:27:34.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1936999999999999e-05 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850436+0000", 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000102631 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850488+0000", 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2169000000000001e-05 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852145+0000", 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016569480000000001 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852268+0000", 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012255999999999999 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128966 3.9 3:9420bb05:::rbd_data.11181500dc3e.000000000000008f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:34.015 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0382569329999995, 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053551420000000002, 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128966 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888748+0000", 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888764+0000", 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6772000000000001e-05 2024-09-13T09:27:34.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888807+0000", 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2827000000000002e-05 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888829+0000", 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1426000000000001e-05 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888965+0000", 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013610599999999999 2024-09-13T09:27:34.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889174+0000", 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020964700000000001 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889456+0000", 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000281403 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.890649+0000", 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011928819999999999 2024-09-13T09:27:34.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893917+0000", 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003268328 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894020+0000", 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103003 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894106+0000", 2024-09-13T09:27:34.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5986e-05 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130683 3.9 3:9160b546:::rbd_data.1118de6ae5af.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.1162867390000004, 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005777351, 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:34.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130683 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810718+0000", 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811313+0000", 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059520999999999999 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811352+0000", 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9453999999999999e-05 2024-09-13T09:27:34.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811369+0000", 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6886999999999999e-05 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811475+0000", 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105845 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811814+0000", 2024-09-13T09:27:34.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033890599999999998 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812483+0000", 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00066937099999999999 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816386+0000", 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039026949999999999 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:34.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816411+0000", 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4916e-05 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816465+0000", 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3835000000000002e-05 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816498+0000", 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3555e-05 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:133033 3.13 e101/94 3:c9a00536:::rbd_data.11182b78662d.000000000000001a:head v 101'625, mlcod=101'623)", 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.77230351600000002, 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042112800000000004, 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:34.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154702+0000", 2024-09-13T09:27:34.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156468+0000", 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017665440000000001 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156503+0000", 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156518+0000", 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5543e-05 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156666+0000", 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014814599999999999 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156733+0000", 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7037000000000002e-05 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.158851+0000", 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021177840000000002 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.158915+0000", 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4343000000000004e-05 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:34.031 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:34.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:34.033 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:34.028+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:34.134 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:34.131+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:34.150 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:34.150 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:34.150 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:34.150 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:34.160 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:34.235 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:34.232+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:34.264 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:34.264 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:34.264 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:34.264 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:34.264 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:34.273 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:34.280 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:27:34.336 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:34.331+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:34.378 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:34.378 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:34.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681570815999997, 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:34.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:34.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681548302000003, 2024-09-13T09:27:34.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681532464999997, 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681335521000001, 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:34.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:34.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681311196999999, 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:34.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:34.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681294737000002, 2024-09-13T09:27:34.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.405 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681278716000001, 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.409 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681228382999997, 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:34.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:34.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.681124808, 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.417 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:34.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671747011999997, 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:34.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671728348000002, 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:34.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671711858000002, 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:34.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:34.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671696073, 2024-09-13T09:27:34.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:34.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671679951999998, 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:34.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671664047, 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:34.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:34.447 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671647577999998, 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671625726000002, 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:34.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671608911, 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:34.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671592613999998, 2024-09-13T09:27:34.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:34.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:34.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.464 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.671577012999997, 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.465 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:34.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.469 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:34.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:34.471 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:34.464+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:34.572 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:34.567+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:34.587 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:34.587 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:34.587 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:34.587 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:34.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:34.672 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:34.667+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:34.751 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:34.751 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:34.751 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:34.760 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:34.773 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:34.772+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:34.871 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509807940999998, 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.872 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:34.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509792482000002, 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:34.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509776402, 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:34.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509760628999999, 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:34.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:34.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:34.888 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509738728000002, 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:34.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:34.892 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509722842000002, 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:34.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:34.896 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509707527000003, 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:34.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:34.900 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509691482000001, 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:34.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:34.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.904 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509675758, 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:34.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509661158, 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.912 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509645730000003, 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:34.916 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.509629844000003, 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:34.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:34.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.160502600999997, 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.160368234000003, 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:34.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:34.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:34.929 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.160347342999998, 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:34.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:34.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.786297738000002, 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:34.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:34.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.785420178000003, 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.938 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:34.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:34.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.785380617999998, 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.942 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.563057146999999, 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:27:34.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:27:34.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:27:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:27:34.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.035843941, 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:34.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:27:34.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:27:34.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:34.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:34.959 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:34.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:34.960 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:34.952+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:35.061 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:35.055+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:35.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:35.074 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:35.074 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:35.075 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:35.075 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:35.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:35.162 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:35.156+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:35.219 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:35.237 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:35.237 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:35.239 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:35.247 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:35.260 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:35.262 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:35.259+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:35.287 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:35.292 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:27:35.329 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:35.352 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:35.359 INFO:teuthology.orchestra.run.smithi063.stdout:4650 write 0x54a89f4 thru 0x54b08c3 (0x7ed0 bytes) 2024-09-13T09:27:35.363 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:35.359+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:35.371 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:35.371 INFO:teuthology.orchestra.run.smithi063.stdout:4652 punch from 0x4c3103f to 0x4c3340e, (0x23cf bytes) 2024-09-13T09:27:35.372 INFO:teuthology.orchestra.run.smithi063.stdout:4654 read 0x1d2af0d thru 0x1d34c8d (0x9d81 bytes) 2024-09-13T09:27:35.372 INFO:teuthology.orchestra.run.smithi063.stdout:4655 punch from 0x121ecbb to 0x1224e18, (0x615d bytes) 2024-09-13T09:27:35.372 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:35.382 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:35.382 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:35.388 INFO:teuthology.orchestra.run.smithi063.stdout:4656 writesame 0x84ce95d thru 0x84d77a2 data_size 0x1(0x8e46 bytes) 2024-09-13T09:27:35.388 INFO:teuthology.orchestra.run.smithi063.stdout:4657 punch from 0x1e4154f to 0x1e4b1cf, (0x9c80 bytes) 2024-09-13T09:27:35.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:35.398 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:35.401 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:35.402 INFO:teuthology.orchestra.run.smithi063.stdout:4659 write 0xacf81bb thru 0xad07d68 (0xfbae bytes) 2024-09-13T09:27:35.424 INFO:teuthology.orchestra.run.smithi063.stdout:4663 write 0xc04a314 thru 0xc051c46 (0x7933 bytes) 2024-09-13T09:27:35.431 INFO:teuthology.orchestra.run.smithi063.stdout:4666 writesame 0x254e4bc thru 0x2556811 data_size 0x1(0x8356 bytes) 2024-09-13T09:27:35.463 INFO:teuthology.orchestra.run.smithi063.stdout:4667 punch from 0x6fde290 to 0x6fdec7a, (0x9ea bytes) 2024-09-13T09:27:35.463 INFO:teuthology.orchestra.run.smithi063.stdout:4669 read 0x6cd6276 thru 0x6cdb7b0 (0x553b bytes) 2024-09-13T09:27:35.463 INFO:teuthology.orchestra.run.smithi063.stdout:4670 writesame 0x8d17e87 thru 0x8d1b4ba data_size 0x2(0x3634 bytes) 2024-09-13T09:27:35.464 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:35.459+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:35.477 INFO:teuthology.orchestra.run.smithi063.stdout:4672 read 0xbc13d18 thru 0xbc22a08 (0xecf1 bytes) 2024-09-13T09:27:35.477 INFO:teuthology.orchestra.run.smithi063.stdout:4675 write 0x90dc891 thru 0x90e9cb0 (0xd420 bytes) 2024-09-13T09:27:35.486 INFO:teuthology.orchestra.run.smithi063.stdout:4678 writesame 0x505e918 thru 0x505ee2f data_size 0x4(0x518 bytes) 2024-09-13T09:27:35.495 INFO:teuthology.orchestra.run.smithi063.stdout:4679 writesame 0x200d14 thru 0x20c30e data_size 0x35(0xb5fb bytes) 2024-09-13T09:27:35.512 INFO:teuthology.orchestra.run.smithi063.stdout:4680 write 0x2dbca7 thru 0x2ddeb1 (0x220b bytes) 2024-09-13T09:27:35.534 INFO:teuthology.orchestra.run.smithi063.stdout:4681 writesame 0x304ee thru 0x334b7 data_size 0x3(0x2fca bytes) 2024-09-13T09:27:35.540 INFO:teuthology.orchestra.run.smithi063.stdout:4682 punch from 0x9582cca to 0x958eada, (0xbe10 bytes) 2024-09-13T09:27:35.559 INFO:teuthology.orchestra.run.smithi063.stdout:4684 trunc from 0xc051c47 to 0x3e0d6e8 2024-09-13T09:27:35.565 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:35.560+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:35.570 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:35.570 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:35.580 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:35.653 INFO:teuthology.orchestra.run.smithi063.stdout:4685 read 0x17789e2 thru 0x1783a4d (0xb06c bytes) 2024-09-13T09:27:35.666 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:35.663+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:35.686 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:35.686 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:35.687 INFO:teuthology.orchestra.run.smithi063.stdout:4687 write 0xdc4fa0b thru 0xdc583c5 (0x89bb bytes) 2024-09-13T09:27:35.694 INFO:teuthology.orchestra.run.smithi063.stdout:4688 write 0xcc998aa thru 0xcca6641 (0xcd98 bytes) 2024-09-13T09:27:35.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:35.707 INFO:teuthology.orchestra.run.smithi063.stdout:4690 trunc from 0xdc583c6 to 0xb89f825 2024-09-13T09:27:35.731 INFO:teuthology.orchestra.run.smithi063.stdout:4691 punch from 0x2bc7ba9 to 0x2bc932f, (0x1786 bytes) 2024-09-13T09:27:35.731 INFO:teuthology.orchestra.run.smithi063.stdout:4692 write 0x6eccbed thru 0x6ecd61b (0xa2f bytes) 2024-09-13T09:27:35.761 INFO:teuthology.orchestra.run.smithi063.stdout:4694 write 0xe1f98d1 thru 0xe2040cd (0xa7fd bytes) 2024-09-13T09:27:35.766 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:35.763+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:35.774 INFO:teuthology.orchestra.run.smithi063.stdout:4696 read 0x39a47d7 thru 0x39a526c (0xa96 bytes) 2024-09-13T09:27:35.774 INFO:teuthology.orchestra.run.smithi063.stdout:4698 read 0xb09627c thru 0xb09ab8f (0x4914 bytes) 2024-09-13T09:27:35.774 INFO:teuthology.orchestra.run.smithi063.stdout:4699 read 0x70da95f thru 0x70df5d4 (0x4c76 bytes) 2024-09-13T09:27:35.774 INFO:teuthology.orchestra.run.smithi063.stdout:4700 write 0x2546c5 thru 0x25b068 (0x69a4 bytes) 2024-09-13T09:27:35.786 INFO:teuthology.orchestra.run.smithi063.stdout:4701 write 0x6c450c9 thru 0x6c4ef52 (0x9e8a bytes) 2024-09-13T09:27:35.797 INFO:teuthology.orchestra.run.smithi063.stdout:4702 trunc from 0xe2040ce to 0xc3b4a63 2024-09-13T09:27:35.809 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:35.809 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:35.817 INFO:teuthology.orchestra.run.smithi063.stdout:4703 read 0xa11f95a thru 0xa1272d1 (0x7978 bytes) 2024-09-13T09:27:35.817 INFO:teuthology.orchestra.run.smithi063.stdout:4704 write 0x589de13 thru 0x58ab3cb (0xd5b9 bytes) 2024-09-13T09:27:35.822 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:35.828 INFO:teuthology.orchestra.run.smithi063.stdout:4706 read 0x63cd8cb thru 0x63d3ffb (0x6731 bytes) 2024-09-13T09:27:35.828 INFO:teuthology.orchestra.run.smithi063.stdout:4707 writesame 0x2afc372 thru 0x2afe862 data_size 0x7(0x24f1 bytes) 2024-09-13T09:27:35.839 INFO:teuthology.orchestra.run.smithi063.stdout:4708 read 0x906256f thru 0x9064207 (0x1c99 bytes) 2024-09-13T09:27:35.839 INFO:teuthology.orchestra.run.smithi063.stdout:4711 punch from 0x50960ad to 0x50a0eff, (0xae52 bytes) 2024-09-13T09:27:35.850 INFO:teuthology.orchestra.run.smithi063.stdout:4713 trunc from 0xc3b4a63 to 0x7a9744a 2024-09-13T09:27:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:4714 read 0x250a286 thru 0x2516b4a (0xc8c5 bytes) 2024-09-13T09:27:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:4715 punch from 0x3eed73c to 0x3ef60ae, (0x8972 bytes) 2024-09-13T09:27:35.862 INFO:teuthology.orchestra.run.smithi063.stdout:4716 write 0x4c02277 thru 0x4c0e9db (0xc765 bytes) 2024-09-13T09:27:35.867 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:35.860+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:4717 read 0x2bdb1e0 thru 0x2be1e53 (0x6c74 bytes) 2024-09-13T09:27:35.871 INFO:teuthology.orchestra.run.smithi063.stdout:4718 write 0x58fd148 thru 0x590411d (0x6fd6 bytes) 2024-09-13T09:27:35.912 INFO:teuthology.orchestra.run.smithi063.stdout:4719 writesame 0xda1de32 thru 0xda2d0a3 data_size 0x1(0xf272 bytes) 2024-09-13T09:27:35.912 INFO:teuthology.orchestra.run.smithi063.stdout:4720 read 0x88f350a thru 0x88fdc7e (0xa775 bytes) 2024-09-13T09:27:35.913 INFO:teuthology.orchestra.run.smithi063.stdout:4721 read 0x2cbe9b8 thru 0x2ccc625 (0xdc6e bytes) 2024-09-13T09:27:35.914 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:35.914 INFO:teuthology.orchestra.run.smithi063.stdout:4727 write 0xa54ae75 thru 0xa552794 (0x7920 bytes) 2024-09-13T09:27:35.921 INFO:teuthology.orchestra.run.smithi063.stdout:4728 writesame 0x7de6546 thru 0x7de8642 data_size 0x5(0x20fd bytes) 2024-09-13T09:27:35.930 INFO:teuthology.orchestra.run.smithi063.stdout:4729 writesame 0x848505e thru 0x8488ec6 data_size 0x1(0x3e69 bytes) 2024-09-13T09:27:35.939 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:35.939 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:35.949 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:35.950 INFO:teuthology.orchestra.run.smithi063.stdout:4730 writesame 0x21a03b2 thru 0x21a0f55 data_size 0xa(0xba4 bytes) 2024-09-13T09:27:35.957 INFO:teuthology.orchestra.run.smithi063.stdout:4732 writesame 0xc978c00 thru 0xc98860c data_size 0x1(0xfa0d bytes) 2024-09-13T09:27:35.968 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:35.964+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:35.986 INFO:teuthology.orchestra.run.smithi063.stdout:4733 trunc from 0xda2d0a4 to 0xd1ece9f 2024-09-13T09:27:35.996 INFO:teuthology.orchestra.run.smithi063.stdout:4734 read 0x464f4ec thru 0x465ec8d (0xf7a2 bytes) 2024-09-13T09:27:35.996 INFO:teuthology.orchestra.run.smithi063.stdout:4735 write 0x39a8c3e thru 0x39abe0c (0x31cf bytes) 2024-09-13T09:27:36.006 INFO:teuthology.orchestra.run.smithi063.stdout:4736 write 0x84d8d56 thru 0x84df040 (0x62eb bytes) 2024-09-13T09:27:36.017 INFO:teuthology.orchestra.run.smithi063.stdout:4737 read 0x6a680b8 thru 0x6a71f2a (0x9e73 bytes) 2024-09-13T09:27:36.019 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:36.019 INFO:teuthology.orchestra.run.smithi063.stdout:4739 read 0x7cfa61f thru 0x7d03f5b (0x993d bytes) 2024-09-13T09:27:36.019 INFO:teuthology.orchestra.run.smithi063.stdout:4740 write 0xd1bffd4 thru 0xd1c9d4f (0x9d7c bytes) 2024-09-13T09:27:36.034 INFO:teuthology.orchestra.run.smithi063.stdout:4741 write 0x54ac8ca thru 0x54bbc19 (0xf350 bytes) 2024-09-13T09:27:36.045 INFO:teuthology.orchestra.run.smithi063.stdout:4743 punch from 0xc6cacde to 0xc6d0d1f, (0x6041 bytes) 2024-09-13T09:27:36.045 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:36.045 INFO:teuthology.orchestra.run.smithi063.stdout:4745 read 0x83e98f6 thru 0x83eaa44 (0x114f bytes) 2024-09-13T09:27:36.045 INFO:teuthology.orchestra.run.smithi063.stdout:4749 clone 36 order 19 su 8192 sc 4 2024-09-13T09:27:36.058 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.058 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.068 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:36.069 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:36.067+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:36.170 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:36.164+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:36.224 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.225 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.235 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:36.280 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:36.276+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:36.348 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.348 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.359 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:36.372 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:36.367+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:36.466 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.466 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.472 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:36.467+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:36.473 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:36.577 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:36.572+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:36.586 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.586 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.598 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:36.684 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:36.672+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:36.776 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:36.768+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:36.778 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.778 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.789 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:36.879 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:36.872+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:36.955 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:36.956 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:36.966 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:37.009 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:37.004+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:37.082 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:37.076+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:37.121 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:37.122 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:37.132 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:37.183 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:37.176+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:37.242 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:37.242 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:37.252 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:37.285 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:37.276+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:37.305 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:27:37.373 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:37.374 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:37.385 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:37.385 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:37.380+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:37.412 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:37.489 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:37.483+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:37.492 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:37.493 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:37.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:37.548 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:37.569 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:37.586 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:37.580+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:37.589 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:37.612 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:37.621 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:37.621 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:37.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:37.636 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:37.657 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:37.677 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:37.688 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:37.680+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:37.697 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:37.700 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:37.700 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:37.700 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:37.700 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:37.700 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:37.701 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:37.701 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:37.701 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:37.756 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:37.756 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:37.756 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:37.756 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:37.756 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:37.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:37.789 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:37.784+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178336576, 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178314061999998, 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178298224999999, 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:37.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178101280999996, 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:37.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:37.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:37.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:37.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178076957000002, 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178060496999997, 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:37.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:37.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.178044475999997, 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:37.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.177994142999999, 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:37.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:37.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:37.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.177890568000002, 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:37.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168512772, 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:37.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168494107999997, 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.924 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:37.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168477617999997, 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.929 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:37.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168461833000002, 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.937 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168445712, 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168429807000003, 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:37.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:37.947 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168413338000001, 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168391485999997, 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:37.952 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168374671000002, 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:37.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:37.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168358374, 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:37.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.965 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.168342772999999, 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:37.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:37.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:37.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:37.970 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:37.972 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:37.972 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:37.967+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:38.072 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:38.067+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:38.101 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:38.101 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:38.101 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:38.173 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:38.172+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:38.224 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.224 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:38.224 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:38.224 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:38.224 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.235 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:38.275 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:38.271+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:38.298 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:38.298 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:38.298 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-13T09:27:38.299 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:27:38.299 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:27:38.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-13T09:27:38.343 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:27:38.374 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013296273999998, 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.378 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013280815000002, 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:38.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013264735, 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:38.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:38.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013248961999999, 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:38.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:38.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013227061000002, 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:38.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:38.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:38.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013211175000002, 2024-09-13T09:27:38.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:38.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013195860000003, 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:38.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013179815000001, 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:38.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013164091, 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.409 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013149491, 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.413 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:38.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013134063000003, 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.013118177000003, 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:38.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.663990933999997, 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:38.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:38.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.663856567000003, 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:38.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.434 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.663835675999998, 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:38.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.289786071000002, 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:38.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:38.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:38.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.288908511000002, 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:38.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.288868950999998, 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:38.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:38.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.066545479999998, 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:27:38.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:27:38.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:38.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.539332274, 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:38.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:27:38.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.466 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:38.468 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.468 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:27:38.468 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.469 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:38.460+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:38.568 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:38.560+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:38.627 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:38.627 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:38.627 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:38.668 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:38.664+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:38.739 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:38.739 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:38.770 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:38.764+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 99.295537319999994, 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.485314058, 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.226571186999998, 2024-09-13T09:27:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.224368849000001, 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.223972248000003, 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.223096863999999, 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.223075881, 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:38.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:38.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.223025368999998, 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:38.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.222995509999997, 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:38.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.172655828000003, 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:38.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:38.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.172597250000003, 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:38.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.172574615000002, 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:38.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:38.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:38.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:38.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.172557652999998, 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:38.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.171685922999998, 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:38.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:38.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.171653288999998, 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:38.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:38.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.171632889999998, 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:38.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.171591204999999, 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.171458543999996, 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.169108090999998, 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.168924095999998, 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:38.952 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:38.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:38.954 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:38.948+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:39.054 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:39.051+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:39.074 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:39.074 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:39.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.074 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3.c9389a13 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:27:39.074 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.67003397799999997, 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.67007419400000001, 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for peered", 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:39.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:27:39.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.079 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:27:39.079 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-13T09:27:39.079 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:39.086 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:39.156 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:39.148+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:39.203 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:39.204 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:39.204 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:39.204 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:39.204 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:39.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:39.256 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:39.251+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:39.326 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:39.326 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:39.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:39.326 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:39.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 116.362286733, 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:39.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.249066988999999, 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:39.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.248668973999997, 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573562097999996, 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:39.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573547165999997, 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:39.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:39.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573514060000001, 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:39.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:39.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:39.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:39.354 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573497766999999, 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:39.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573481651999998, 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:39.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:39.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573450610999998, 2024-09-13T09:27:39.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:39.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:39.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.573418988, 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:39.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:39.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86979 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 760832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.562505338999998, 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054560900000000002, 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86979 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760796+0000", 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760798+0000", 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.243e-06 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760803+0000", 2024-09-13T09:27:39.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.271e-06 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760807+0000", 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3019999999999996e-06 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815245+0000", 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054437946000000001 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815284+0000", 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8127999999999998e-05 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.815357+0000", 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3434000000000002e-05 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560470926999997, 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:39.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560453871, 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:39.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:39.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560437702999998, 2024-09-13T09:27:39.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.386 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:39.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560224040999998, 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:39.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:39.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560204730000002, 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:39.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:39.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560188337, 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.400 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560172674999997, 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:39.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.560146320999998, 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.0316335939999997, 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:39.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:39.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:27:39.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:27:39.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:39.420 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:39.422 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:39.422 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:39.416+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:39.525 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:39.523+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:39.537 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:39.537 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:39.548 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:39.627 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:39.620+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:39.656 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:39.656 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:39.666 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:39.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:39.727 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:39.723+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:39.808 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:39.808 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 35.064891410999998, 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:39.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:39.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:39.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:39.816 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124586 3.b 3:d583c8fe:::rbd_data.1118fc9bb66a.0000000000000005:1e [sparse-read 131072~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.667155430999998, 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062408630000000001, 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.817 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124586 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135390+0000", 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135389+0000", 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135396+0000", 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7309999999999996e-06 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135404+0000", 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5369999999999997e-06 2024-09-13T09:27:39.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135411+0000", 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140323+0000", 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049127659999999998 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.140390+0000", 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6631999999999995e-05 2024-09-13T09:27:39.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.141630+0000", 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012403480000000001 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124592 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.667081603, 2024-09-13T09:27:39.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0076439699999999999, 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124592 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135463+0000", 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135469+0000", 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:27:39.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135500+0000", 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1149000000000003e-05 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135513+0000", 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2771e-05 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142205+0000", 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066924239999999998 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.142329+0000", 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012366100000000001 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143107+0000", 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077859999999999995 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.825 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124596 3.b 3:d0a4f1be:::rbd_data.1118fc9bb66a.0000000000000009:1e [sparse-read 131072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.666969793, 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078700200000000001, 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124596 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.826 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135575+0000", 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135574+0000", 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135581+0000", 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3280000000000002e-06 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135597+0000", 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5996999999999999e-05 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.135608+0000", 2024-09-13T09:27:39.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.166e-05 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143162+0000", 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0075534349999999998 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143306+0000", 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014448900000000001 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.143445+0000", 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013898200000000001 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124608 3.7 3:e4e5f482:::rbd_data.1118f3eff690.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.657491876999998, 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062754919999999997, 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.830 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124608 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145053+0000", 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145052+0000", 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145531+0000", 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00047855199999999998 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145542+0000", 2024-09-13T09:27:39.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0961000000000001e-05 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145551+0000", 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145661+0000", 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000109726 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.145992+0000", 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033164000000000002 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.146845+0000", 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00085247199999999995 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.148693+0000", 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018483460000000001 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151235+0000", 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002542159 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151297+0000", 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1710000000000004e-05 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.151329+0000", 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1730000000000003e-05 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.835 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124700 3.1e 3:78e395d8:::rbd_data.1118f3eff690.0000000000000080:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.500248681999999, 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011658599, 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.836 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124700 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302294+0000", 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302307+0000", 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3602e-05 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302336+0000", 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8116999999999999e-05 2024-09-13T09:27:39.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302348+0000", 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2792000000000001e-05 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302394+0000", 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5797000000000002e-05 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302505+0000", 2024-09-13T09:27:39.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111286 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.304334+0000", 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828325 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313641+0000", 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093073449999999999 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313729+0000", 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8305000000000002e-05 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313834+0000", 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010443899999999999 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313955+0000", 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121142 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.841 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:124700 3.1e e78/77 ondisk, result = 0)", 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.494363944, 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057895580000000002, 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:39.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308181+0000", 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308180+0000", 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308188+0000", 2024-09-13T09:27:39.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7800000000000001e-06 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308203+0000", 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4977000000000001e-05 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.308216+0000", 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2855e-05 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313698+0000", 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0054822960000000002 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313716+0000", 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8167000000000001e-05 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313971+0000", 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025462500000000002 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:125238 3.18 3:1cc4019a:::rbd_data.1118bda513e1.0000000000000005:head [stat,write 16509626~3298] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.761225186000001, 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0065355600000000002, 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 125238 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041320+0000", 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041315+0000", 2024-09-13T09:27:39.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999957 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041336+0000", 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1073e-05 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041373+0000", 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7039000000000002e-05 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041394+0000", 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0477000000000001e-05 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041475+0000", 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0952999999999996e-05 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op must wait for map", 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.041481+0000", 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0569999999999998e-06 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045492+0000", 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040115339999999998 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045585+0000", 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2255000000000004e-05 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.045728+0000", 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014281499999999999 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047526+0000", 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017983400000000001 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047681+0000", 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015514200000000001 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047777+0000", 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6415000000000004e-05 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.047855+0000", 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7985999999999995e-05 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126514 3.1e 3:7f6180ef:::rbd_object_map.1118413dc0c1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.575547259, 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0077678299999999999, 2024-09-13T09:27:39.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126514 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226998+0000", 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.226997+0000", 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227004+0000", 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.187e-06 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227027+0000", 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3254000000000001e-05 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227034+0000", 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6479999999999997e-06 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227078+0000", 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227182+0000", 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104095 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227687+0000", 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00050565500000000004 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.233076+0000", 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053885319999999997 2024-09-13T09:27:39.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234572+0000", 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014962490000000001 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234697+0000", 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000124626 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234766+0000", 2024-09-13T09:27:39.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8842000000000003e-05 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126516 3.1e 3:7f6180ef:::rbd_object_map.1118413dc0c1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.575170280000002, 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078524679999999996, 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126516 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227375+0000", 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227373+0000", 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227386+0000", 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2916e-05 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227409+0000", 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3268000000000002e-05 2024-09-13T09:27:39.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.227425+0000", 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5798e-05 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.230833+0000", 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034084919999999999 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.231022+0000", 2024-09-13T09:27:39.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000188763 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.231925+0000", 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00090269899999999997 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.234795+0000", 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028700589999999999 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235100+0000", 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030537400000000002 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235153+0000", 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2222999999999998e-05 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:16.235227+0000", 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4719000000000001e-05 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128961 3.b 3:d06f0007:::rbd_data.11181500dc3e.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.955278462000001, 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010200722000000001, 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128961 2024-09-13T09:27:39.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847265+0000", 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847762+0000", 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00049738799999999998 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847787+0000", 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4451999999999999e-05 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847799+0000", 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2140999999999999e-05 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847886+0000", 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7020000000000004e-05 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.848076+0000", 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019032700000000001 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852181+0000", 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041049830000000004 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857348+0000", 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051670809999999996 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857367+0000", 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8533e-05 2024-09-13T09:27:39.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857428+0000", 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1334999999999995e-05 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857467+0000", 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9286000000000003e-05 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129048 3.b 3:d1c06694:::rbd_data.11181500dc3e.0000000000000088:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.760241968999999, 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014253025000000001, 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129048 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042302+0000", 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042309+0000", 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.601e-06 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042322+0000", 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3529999999999999e-05 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042327+0000", 2024-09-13T09:27:39.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042399+0000", 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1208e-05 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042524+0000", 2024-09-13T09:27:39.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012532799999999999 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.044815+0000", 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022907729999999999 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056394+0000", 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011579271 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056445+0000", 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0994e-05 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056500+0000", 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5442999999999998e-05 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056556+0000", 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5702000000000003e-05 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129113 3.7 3:e0643571:::rbd_data.11181500dc3e.000000000000016e:head [set-alloc-hint object_size 524288 write_size 524288,write 98304~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.669193852999999, 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057129440000000002, 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129113 2024-09-13T09:27:39.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133351+0000", 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133348+0000", 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133380+0000", 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2150000000000002e-05 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133409+0000", 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8924e-05 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133423+0000", 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3773e-05 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133527+0000", 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010395600000000001 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.133780+0000", 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025285500000000002 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134012+0000", 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023174799999999999 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.137751+0000", 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0037395369999999998 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.138980+0000", 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001228282 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.139035+0000", 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5652999999999997e-05 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.139064+0000", 2024-09-13T09:27:39.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8969e-05 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129140 3.7 3:e0643571:::rbd_data.11181500dc3e.000000000000016e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.591245326999999, 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086215130000000008, 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.882 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129140 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211300+0000", 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211298+0000", 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.211307+0000", 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6829999999999997e-06 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212397+0000", 2024-09-13T09:27:39.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010900009999999999 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212409+0000", 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2746e-05 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.212490+0000", 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0668999999999996e-05 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.213419+0000", 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00092909700000000004 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.213568+0000", 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000148531 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.214326+0000", 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00075828500000000004 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219843+0000", 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055167430000000002 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219875+0000", 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1869000000000003e-05 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.219921+0000", 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6560000000000001e-05 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.887 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130752 3.2 3:45d3584e:::rbd_data.1118de6ae5af.000000000000004e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.988479+0000", 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.814065577000001, 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012273492, 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.888 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130752 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988479+0000", 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988479+0000", 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988477+0000", 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988494+0000", 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6617e-05 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988526+0000", 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2660999999999998e-05 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988540+0000", 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3994e-05 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988627+0000", 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6427000000000004e-05 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988807+0000", 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018076499999999999 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:39.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.990749+0000", 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019417600000000001 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000452+0000", 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0097028979999999997 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000488+0000", 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5644e-05 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000635+0000", 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014702199999999999 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000753+0000", 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118272 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.893 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130776 3.8 3:1727d39e:::rbd_data.1118de6ae5af.00000000000000c4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.071035+0000", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.73150985, 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015911854999999999, 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130776 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071035+0000", 2024-09-13T09:27:39.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071035+0000", 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071032+0000", 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071049+0000", 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.702e-05 2024-09-13T09:27:39.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071082+0000", 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3043999999999997e-05 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071098+0000", 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5339999999999999e-05 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071183+0000", 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5415999999999997e-05 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071389+0000", 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000206269 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.073943+0000", 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025537950000000002 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086726+0000", 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012782401000000001 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086802+0000", 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086885+0000", 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3357999999999997e-05 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086947+0000", 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2147e-05 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130830 3.2 3:45d3584e:::rbd_data.1118de6ae5af.000000000000004e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.218689+0000", 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.583856469000001, 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0060544379999999997, 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.899 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130830 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218689+0000", 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218689+0000", 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218686+0000", 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218700+0000", 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3983e-05 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218724+0000", 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3937000000000001e-05 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218734+0000", 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0635000000000001e-05 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218882+0000", 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014721499999999999 2024-09-13T09:27:39.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219080+0000", 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019835 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219331+0000", 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025116 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.222663+0000", 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033318200000000001 2024-09-13T09:27:39.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.224583+0000", 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019204809999999999 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.224650+0000", 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6415999999999996e-05 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.224743+0000", 2024-09-13T09:27:39.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3158000000000005e-05 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130834 3.18 3:1a65d6e2:::rbd_data.1118de6ae5af.000000000000006d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.218770+0000", 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.583775434, 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0083168300000000007, 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.905 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130834 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218770+0000", 2024-09-13T09:27:39.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218770+0000", 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218769+0000", 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218773+0000", 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3420000000000001e-06 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218792+0000", 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9034000000000001e-05 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218799+0000", 2024-09-13T09:27:39.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218858+0000", 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9522999999999999e-05 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218959+0000", 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010099600000000001 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219080+0000", 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012116399999999999 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.220693+0000", 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001612182 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.226995+0000", 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063019160000000003 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227047+0000", 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227086+0000", 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9654999999999999e-05 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.911 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:132654 3.18 e101/75 ondisk, result = 0)", 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:32.880970+0000", 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.9215747189999997, 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010354965000000001, 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880970+0000", 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880970+0000", 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880969+0000", 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880977+0000", 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5859999999999999e-06 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880989+0000", 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880997+0000", 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2970000000000004e-06 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.891204+0000", 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010207188000000001 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.891213+0000", 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6219999999999998e-06 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.891325+0000", 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112281 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.915 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:132774 3.7 3:e51b177d:::rbd_data.11182b78662d.0000000000000055:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101)", 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:33.007583+0000", 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.7949617519999999, 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080294240000000003, 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 132774 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:39.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007583+0000", 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007583+0000", 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007582+0000", 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007588+0000", 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8139999999999997e-06 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007614+0000", 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5840999999999999e-05 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007624+0000", 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.944e-06 2024-09-13T09:27:39.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007734+0000", 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110159 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.007866+0000", 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013150099999999999 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.008039+0000", 2024-09-13T09:27:39.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017286999999999999 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.009535+0000", 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014967559999999999 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.015463+0000", 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059281710000000003 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.015576+0000", 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112409 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.015613+0000", 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6808999999999999e-05 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:39.921 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:39.923 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:39.924 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:39.924 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:39.924 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:39.924 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:39.916+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:39.931 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:39.955 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:39.982 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:40.005 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:40.023 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:40.019+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:40.029 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:40.031 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:40.031 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:40.031 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.032 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.042 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:40.124 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:40.119+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:40.152 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.152 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.164 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:40.225 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:40.223+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:40.226 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone35 from 0xd1ece9f (overlap 0xb37a2c) to 0x59bd16 2024-09-13T09:27:40.278 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.278 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.287 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:40.326 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:40.320+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:40.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:27:40.387 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.387 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.397 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:40.444 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:40.440+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:40.529 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:40.524+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:40.556 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.557 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.563 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:40.630 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:40.627+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:40.662 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.663 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.672 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:40.731 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:40.724+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:40.780 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.780 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.790 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:40.831 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:40.824+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:40.893 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:40.893 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:40.903 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:40.933 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:40.931+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:41.034 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:41.031+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:41.057 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.057 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:41.135 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:41.128+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:41.166 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:27:41.192 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.192 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:41.235 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:41.228+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:41.309 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.309 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.319 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:41.335 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T09:27:41.336 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:41.332+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:41.420 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.420 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:41.436 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:41.431+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:41.538 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:41.535+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:41.538 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.538 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.548 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:41.639 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:41.635+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:41.655 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.655 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:41.672 INFO:teuthology.orchestra.run.smithi063.stdout:4750 read 0x67c4a8b thru 0x67d3834 (0xedaa bytes) 2024-09-13T09:27:41.702 INFO:teuthology.orchestra.run.smithi063.stdout:4751 write 0xedc3582 thru 0xedcc062 (0x8ae1 bytes) 2024-09-13T09:27:41.711 INFO:teuthology.orchestra.run.smithi063.stdout:4752 read 0x64d874 thru 0x65241d (0x4baa bytes) 2024-09-13T09:27:41.718 INFO:teuthology.orchestra.run.smithi063.stdout:4753 read 0x56ba542 thru 0x56c12c1 (0x6d80 bytes) 2024-09-13T09:27:41.723 INFO:teuthology.orchestra.run.smithi063.stdout:4754 write 0xec1a500 thru 0xec22bbc (0x86bd bytes) 2024-09-13T09:27:41.727 INFO:teuthology.orchestra.run.smithi063.stdout:4755 writesame 0xd9f3ef thru 0xdac98d data_size 0x1(0xd59f bytes) 2024-09-13T09:27:41.740 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:41.732+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:41.765 INFO:teuthology.orchestra.run.smithi063.stdout:4756 trunc from 0xedcc063 to 0xa37c844 2024-09-13T09:27:41.789 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.789 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.799 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:41.814 INFO:teuthology.orchestra.run.smithi063.stdout:4757 trunc from 0xa37c844 to 0x288a7b 2024-09-13T09:27:41.841 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:41.836+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:41.875 INFO:teuthology.orchestra.run.smithi063.stdout:4758 writesame 0x49a1fc6 thru 0x49b1f4a data_size 0x1(0xff85 bytes) 2024-09-13T09:27:41.893 INFO:teuthology.orchestra.run.smithi063.stdout:4760 write 0x8808acb thru 0x88094da (0xa10 bytes) 2024-09-13T09:27:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:41.909 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:41.920 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:41.922 INFO:teuthology.orchestra.run.smithi063.stdout:4761 writesame 0xd190e96 thru 0xd1977fc data_size 0xb(0x6967 bytes) 2024-09-13T09:27:41.923 INFO:teuthology.orchestra.run.smithi063.stdout:4762 write 0x48f8f02 thru 0x4904ccf (0xbdce bytes) 2024-09-13T09:27:41.935 INFO:teuthology.orchestra.run.smithi063.stdout:4763 punch from 0x831c0 to 0x8901e, (0x5e5e bytes) 2024-09-13T09:27:41.935 INFO:teuthology.orchestra.run.smithi063.stdout:4764 punch from 0x566d8c9 to 0x5672901, (0x5038 bytes) 2024-09-13T09:27:41.935 INFO:teuthology.orchestra.run.smithi063.stdout:4765 read 0x9c5ffcc thru 0x9c68808 (0x883d bytes) 2024-09-13T09:27:41.935 INFO:teuthology.orchestra.run.smithi063.stdout:4767 read 0xad221e1 thru 0xad264a3 (0x42c3 bytes) 2024-09-13T09:27:41.935 INFO:teuthology.orchestra.run.smithi063.stdout:4768 writesame 0xb78d49d thru 0xb79a98d data_size 0x1b(0xd4f1 bytes) 2024-09-13T09:27:41.941 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:41.936+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:41.947 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:41.948 INFO:teuthology.orchestra.run.smithi063.stdout:4770 punch from 0xd063914 to 0xd0687b4, (0x4ea0 bytes) 2024-09-13T09:27:41.948 INFO:teuthology.orchestra.run.smithi063.stdout:4771 read 0x9dd6bb5 thru 0x9de1dc6 (0xb212 bytes) 2024-09-13T09:27:41.948 INFO:teuthology.orchestra.run.smithi063.stdout:4772 writesame 0x1e55f8b thru 0x1e60cad data_size 0x1(0xad23 bytes) 2024-09-13T09:27:41.974 INFO:teuthology.orchestra.run.smithi063.stdout:4774 read 0xaeece09 thru 0xaef80d5 (0xb2cd bytes) 2024-09-13T09:27:41.974 INFO:teuthology.orchestra.run.smithi063.stdout:4775 punch from 0x2ee5140 to 0x2ef01bf, (0xb07f bytes) 2024-09-13T09:27:41.974 INFO:teuthology.orchestra.run.smithi063.stdout:4776 writesame 0x288a815 thru 0x2897125 data_size 0x1(0xc911 bytes) 2024-09-13T09:27:41.997 INFO:teuthology.orchestra.run.smithi063.stdout:4777 punch from 0x7a59c1f to 0x7a6737f, (0xd760 bytes) 2024-09-13T09:27:41.997 INFO:teuthology.orchestra.run.smithi063.stdout:4779 read 0x914de01 thru 0x9157bfc (0x9dfc bytes) 2024-09-13T09:27:41.997 INFO:teuthology.orchestra.run.smithi063.stdout:4781 read 0x8a26df thru 0x8b04cc (0xddee bytes) 2024-09-13T09:27:42.008 INFO:teuthology.orchestra.run.smithi063.stdout:4782 trunc from 0xd1977fd to 0x6fa15e1 2024-09-13T09:27:42.028 INFO:teuthology.orchestra.run.smithi063.stdout:4784 punch from 0x52e70f2 to 0x52f5fa6, (0xeeb4 bytes) 2024-09-13T09:27:42.028 INFO:teuthology.orchestra.run.smithi063.stdout:4787 writesame 0x2165179 thru 0x216d1ad data_size 0x1(0x8035 bytes) 2024-09-13T09:27:42.029 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:42.029 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:42.041 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:42.043 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:42.036+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:42.044 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:42.047 INFO:teuthology.orchestra.run.smithi063.stdout:4788 punch from 0x2b5d4b3 to 0x2b6d2f0, (0xfe3d bytes) 2024-09-13T09:27:42.070 INFO:teuthology.orchestra.run.smithi063.stdout:4789 writesame 0xd954d74 thru 0xd961df0 data_size 0x1(0xd07d bytes) 2024-09-13T09:27:42.070 INFO:teuthology.orchestra.run.smithi063.stdout:4790 read 0x76620ab thru 0x766ebdf (0xcb35 bytes) 2024-09-13T09:27:42.070 INFO:teuthology.orchestra.run.smithi063.stdout:4791 writesame 0x2faebe thru 0x2fdfe0 data_size 0x7(0x3123 bytes) 2024-09-13T09:27:42.074 INFO:teuthology.orchestra.run.smithi063.stdout:4792 read 0xbe9ce3d thru 0xbea20c8 (0x528c bytes) 2024-09-13T09:27:42.074 INFO:teuthology.orchestra.run.smithi063.stdout:4793 write 0x2976419 thru 0x298569f (0xf287 bytes) 2024-09-13T09:27:42.078 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.078 INFO:teuthology.orchestra.run.smithi063.stdout:4797 write 0xc599163 thru 0xc59a48a (0x1328 bytes) 2024-09-13T09:27:42.094 INFO:teuthology.orchestra.run.smithi063.stdout:4798 writesame 0xddae45d thru 0xddb4877 data_size 0x7(0x641b bytes) 2024-09-13T09:27:42.094 INFO:teuthology.orchestra.run.smithi063.stdout:4799 read 0x4b0991e thru 0x4b0e242 (0x4925 bytes) 2024-09-13T09:27:42.094 INFO:teuthology.orchestra.run.smithi063.stdout:4800 writesame 0xbd81c52 thru 0xbd89f71 data_size 0x10(0x8320 bytes) 2024-09-13T09:27:42.105 INFO:teuthology.orchestra.run.smithi063.stdout:4801 punch from 0x2d85dfe to 0x2d8631b, (0x51d bytes) 2024-09-13T09:27:42.106 INFO:teuthology.orchestra.run.smithi063.stdout:4802 writesame 0x675429f thru 0x6763fc2 data_size 0x22(0xfd24 bytes) 2024-09-13T09:27:42.121 INFO:teuthology.orchestra.run.smithi063.stdout:4803 read 0xd10c411 thru 0xd112376 (0x5f66 bytes) 2024-09-13T09:27:42.121 INFO:teuthology.orchestra.run.smithi063.stdout:4804 read 0x95ef04c thru 0x95f97ea (0xa79f bytes) 2024-09-13T09:27:42.121 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.121 INFO:teuthology.orchestra.run.smithi063.stdout:4807 write 0x53bb595 thru 0x53c4ade (0x954a bytes) 2024-09-13T09:27:42.134 INFO:teuthology.orchestra.run.smithi063.stdout:4808 read 0x2a56822 thru 0x2a5db54 (0x7333 bytes) 2024-09-13T09:27:42.134 INFO:teuthology.orchestra.run.smithi063.stdout:4809 read 0x7ac28 thru 0x879ed (0xcdc6 bytes) 2024-09-13T09:27:42.142 INFO:teuthology.orchestra.run.smithi063.stdout:4810 writesame 0x7de9944 thru 0x7df96d2 data_size 0x4d(0xfd8f bytes) 2024-09-13T09:27:42.143 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:42.136+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:42.151 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:42.151 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:4811 read 0x27206c4 thru 0x2723d69 (0x36a6 bytes) 2024-09-13T09:27:42.157 INFO:teuthology.orchestra.run.smithi063.stdout:4813 write 0xc90bb60 thru 0xc910e68 (0x5309 bytes) 2024-09-13T09:27:42.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:42.166 INFO:teuthology.orchestra.run.smithi063.stdout:4815 writesame 0xcee3b0 thru 0xcf9647 data_size 0x7f(0xb298 bytes) 2024-09-13T09:27:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:4817 punch from 0xa2cdaa3 to 0xa2dcb8d, (0xf0ea bytes) 2024-09-13T09:27:42.172 INFO:teuthology.orchestra.run.smithi063.stdout:4820 write 0xacd90c7 thru 0xace30e6 (0xa020 bytes) 2024-09-13T09:27:42.178 INFO:teuthology.orchestra.run.smithi063.stdout:4821 punch from 0xb2943e to 0xb2fa09, (0x65cb bytes) 2024-09-13T09:27:42.178 INFO:teuthology.orchestra.run.smithi063.stdout:4822 writesame 0x4e3d6a6 thru 0x4e44e61 data_size 0x1(0x77bc bytes) 2024-09-13T09:27:42.192 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:42.193 INFO:teuthology.orchestra.run.smithi063.stdout:4823 read 0x58944bc thru 0x5899377 (0x4ebc bytes) 2024-09-13T09:27:42.193 INFO:teuthology.orchestra.run.smithi063.stdout:4824 punch from 0x87e41c5 to 0x87ec034, (0x7e6f bytes) 2024-09-13T09:27:42.193 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.193 INFO:teuthology.orchestra.run.smithi063.stdout:4827 write 0xac82229 thru 0xac84048 (0x1e20 bytes) 2024-09-13T09:27:42.203 INFO:teuthology.orchestra.run.smithi063.stdout:4829 write 0xe9630e0 thru 0xe964732 (0x1653 bytes) 2024-09-13T09:27:42.219 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:42.220 INFO:teuthology.orchestra.run.smithi063.stdout:4830 trunc from 0xe964733 to 0x9070ef8 2024-09-13T09:27:42.244 INFO:teuthology.orchestra.run.smithi063.stdout:4833 write 0x4fa26dd thru 0x4fa77d6 (0x50fa bytes) 2024-09-13T09:27:42.245 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:42.245 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:42.239+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:42.249 INFO:teuthology.orchestra.run.smithi063.stdout:4836 writesame 0x7a17e4a thru 0x7a26d39 data_size 0x8(0xeef0 bytes) 2024-09-13T09:27:42.261 INFO:teuthology.orchestra.run.smithi063.stdout:4837 write 0x26c04a2 thru 0x26c7d70 (0x78cf bytes) 2024-09-13T09:27:42.269 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:42.269 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:42.273 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:42.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:42.289 INFO:teuthology.orchestra.run.smithi063.stdout:4838 write 0x9a27afb thru 0x9a2b107 (0x360d bytes) 2024-09-13T09:27:42.301 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:42.316 INFO:teuthology.orchestra.run.smithi063.stdout:4839 trunc from 0x9a2b108 to 0xfc8445 2024-09-13T09:27:42.332 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:42.345 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:42.340+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:42.353 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:42.354 INFO:teuthology.orchestra.run.smithi063.stdout:4840 read 0xccd55c thru 0xccf3f3 (0x1e98 bytes) 2024-09-13T09:27:42.374 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:42.376 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:42.376 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:42.376 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:42.376 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:42.377 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:42.377 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:42.377 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:42.377 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:42.377 INFO:teuthology.orchestra.run.smithi063.stdout:4841 write 0x8918100 thru 0x891dcde (0x5bdf bytes) 2024-09-13T09:27:42.385 INFO:teuthology.orchestra.run.smithi063.stdout:4842 write 0x5cb0161 thru 0x5cbc80a (0xc6aa bytes) 2024-09-13T09:27:42.395 INFO:teuthology.orchestra.run.smithi063.stdout:4843 write 0x4733ff5 thru 0x4737ef0 (0x3efc bytes) 2024-09-13T09:27:42.400 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:42.401 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:42.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:42.424 INFO:teuthology.orchestra.run.smithi063.stdout:4844 writesame 0xd69cd90 thru 0xd6a532b data_size 0x22(0x859c bytes) 2024-09-13T09:27:42.424 INFO:teuthology.orchestra.run.smithi063.stdout:4845 read 0x88738d7 thru 0x8880325 (0xca4f bytes) 2024-09-13T09:27:42.425 INFO:teuthology.orchestra.run.smithi063.stdout:4846 write 0x200ea13 thru 0x20100c0 (0x16ae bytes) 2024-09-13T09:27:42.433 INFO:teuthology.orchestra.run.smithi063.stdout:4847 write 0xb03d9d6 thru 0xb046187 (0x87b2 bytes) 2024-09-13T09:27:42.445 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:42.443+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:42.447 INFO:teuthology.orchestra.run.smithi063.stdout:4848 write 0x479db82 thru 0x47a1004 (0x3483 bytes) 2024-09-13T09:27:42.452 INFO:teuthology.orchestra.run.smithi063.stdout:4849 punch from 0x50efd86 to 0x50fd655, (0xd8cf bytes) 2024-09-13T09:27:42.453 INFO:teuthology.orchestra.run.smithi063.stdout:4851 read 0x199f2b6 thru 0x19a6f4f (0x7c9a bytes) 2024-09-13T09:27:42.453 INFO:teuthology.orchestra.run.smithi063.stdout:4853 read 0x13f0a0 thru 0x143f49 (0x4eaa bytes) 2024-09-13T09:27:42.464 INFO:teuthology.orchestra.run.smithi063.stdout:4854 trunc from 0xd6a532c to 0x93e9ab2 2024-09-13T09:27:42.486 INFO:teuthology.orchestra.run.smithi063.stdout:4855 read 0x7be2bb4 thru 0x7be84ce (0x591b bytes) 2024-09-13T09:27:42.509 INFO:teuthology.orchestra.run.smithi063.stdout:4858 write 0xd968d84 thru 0xd9707d8 (0x7a55 bytes) 2024-09-13T09:27:42.519 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:42.519 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 119.554506308, 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:42.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:42.524 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.441286564000002, 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.440888549, 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.529 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:42.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.533 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765781672999999, 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:42.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765766741, 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:42.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:42.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:42.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:42.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765733634999997, 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:42.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:42.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765717342000002, 2024-09-13T09:27:42.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:42.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:42.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765701227000001, 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765670186000001, 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.557 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:42.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.765638563000003, 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:42.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752690502, 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:42.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:42.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:42.570 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752673446000003, 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:42.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752657278000001, 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:42.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752443616000001, 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:42.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752424304999998, 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:42.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:42.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:42.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:42.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:42.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:42.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:42.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:42.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.594 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.594 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:42.594 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752407912000002, 2024-09-13T09:27:42.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:42.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.75239225, 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:42.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:42.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:42.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.752365896000001, 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.607 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:42.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:42.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2238531689999999, 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:27:42.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:27:42.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:27:42.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:27:42.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:27:42.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:42.623 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1119390539999996, 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:27:42.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:42.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:27:42.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:27:42.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:27:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:42.632 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:42.634 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:42.634 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:42.627+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:42.635 INFO:teuthology.orchestra.run.smithi063.stdout:4859 write 0xc710f0 thru 0xc76fab (0x5ebc bytes) 2024-09-13T09:27:42.635 INFO:teuthology.orchestra.run.smithi063.stdout:4860 read 0x4dbac0b thru 0x4dbddfb (0x31f1 bytes) 2024-09-13T09:27:42.635 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4863 writesame 0x9066403 thru 0x9068018 data_size 0x5(0x1c16 bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4864 punch from 0xb3ff1b8 to 0xb3ff2c6, (0x10e bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4869 write 0x7599087 thru 0x75993e0 (0x35a bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4870 trunc from 0xd9707d9 to 0x7f43446 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4871 punch from 0x772c231 to 0x773017e, (0x3f4d bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4872 read 0x2cbff96 thru 0x2cc9a1e (0x9a89 bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4873 read 0x5505036 thru 0x5511dca (0xcd95 bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4874 punch from 0x4cef2d1 to 0x4cfd261, (0xdf90 bytes) 2024-09-13T09:27:42.636 INFO:teuthology.orchestra.run.smithi063.stdout:4876 write 0xec1a5e5 thru 0xec24ff6 (0xaa12 bytes) 2024-09-13T09:27:42.637 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.637 INFO:teuthology.orchestra.run.smithi063.stdout:4879 punch from 0x4230277 to 0x423197e, (0x1707 bytes) 2024-09-13T09:27:42.637 INFO:teuthology.orchestra.run.smithi063.stdout:4880 write 0x5bc6ba5 thru 0x5bc7ce0 (0x113c bytes) 2024-09-13T09:27:42.637 INFO:teuthology.orchestra.run.smithi063.stdout:4881 write 0x3425283 thru 0x3426fac (0x1d2a bytes) 2024-09-13T09:27:42.637 INFO:teuthology.orchestra.run.smithi063.stdout:4882 write 0x743ee64 thru 0x7447962 (0x8aff bytes) 2024-09-13T09:27:42.639 INFO:teuthology.orchestra.run.smithi063.stdout:4883 punch from 0x2f1072a to 0x2f1b713, (0xafe9 bytes) 2024-09-13T09:27:42.639 INFO:teuthology.orchestra.run.smithi063.stdout:4884 write 0x7a9d161 thru 0x7a9ea7b (0x191b bytes) 2024-09-13T09:27:42.648 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.648 INFO:teuthology.orchestra.run.smithi063.stdout:4887 writesame 0x1b218b thru 0x1bb51b data_size 0x1(0x9391 bytes) 2024-09-13T09:27:42.663 INFO:teuthology.orchestra.run.smithi063.stdout:4888 writesame 0x21ce1f0 thru 0x21d024f data_size 0x8(0x2060 bytes) 2024-09-13T09:27:42.666 INFO:teuthology.orchestra.run.smithi063.stdout:4889 read 0x630555d thru 0x6311243 (0xbce7 bytes) 2024-09-13T09:27:42.666 INFO:teuthology.orchestra.run.smithi063.stdout:4891 write 0xdbf8a41 thru 0xdc06e18 (0xe3d8 bytes) 2024-09-13T09:27:42.685 INFO:teuthology.orchestra.run.smithi063.stdout:4892 writesame 0xc34eb6e thru 0xc353db6 data_size 0xb(0x5249 bytes) 2024-09-13T09:27:42.695 INFO:teuthology.orchestra.run.smithi063.stdout:4893 writesame 0x98c7b4e thru 0x98d42da data_size 0x11(0xc78d bytes) 2024-09-13T09:27:42.706 INFO:teuthology.orchestra.run.smithi063.stdout:4894 writesame 0xd345b07 thru 0xd350d96 data_size 0x8(0xb290 bytes) 2024-09-13T09:27:42.717 INFO:teuthology.orchestra.run.smithi063.stdout:4895 write 0x52e6692 thru 0x52eda44 (0x73b3 bytes) 2024-09-13T09:27:42.731 INFO:teuthology.orchestra.run.smithi063.stdout:4898 writesame 0x6c8504b thru 0x6c8e3fb data_size 0x3(0x93b1 bytes) 2024-09-13T09:27:42.734 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:42.731+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:42.746 INFO:teuthology.orchestra.run.smithi063.stdout:4899 writesame 0x7d5988c thru 0x7d621ff data_size 0x26(0x8974 bytes) 2024-09-13T09:27:42.761 INFO:teuthology.orchestra.run.smithi063.stdout:4902 punch from 0x29b93db to 0x29c10d0, (0x7cf5 bytes) 2024-09-13T09:27:42.761 INFO:teuthology.orchestra.run.smithi063.stdout:4903 read 0x8da8d58 thru 0x8db8133 (0xf3dc bytes) 2024-09-13T09:27:42.761 INFO:teuthology.orchestra.run.smithi063.stdout:4906 write 0xad3e834 thru 0xad46559 (0x7d26 bytes) 2024-09-13T09:27:42.761 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:42.761 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:42.761 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:42.762 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:42.771 INFO:teuthology.orchestra.run.smithi063.stdout:4907 write 0xa07bf47 thru 0xa082832 (0x68ec bytes) 2024-09-13T09:27:42.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:42.780 INFO:teuthology.orchestra.run.smithi063.stdout:4908 write 0xcd48832 thru 0xcd54776 (0xbf45 bytes) 2024-09-13T09:27:42.789 INFO:teuthology.orchestra.run.smithi063.stdout:4912 read 0x79f470d thru 0x79f84f8 (0x3dec bytes) 2024-09-13T09:27:42.790 INFO:teuthology.orchestra.run.smithi063.stdout:4913 write 0xd602779 thru 0xd60b041 (0x88c9 bytes) 2024-09-13T09:27:42.798 INFO:teuthology.orchestra.run.smithi063.stdout:4914 punch from 0xb87174e to 0xb87a5f3, (0x8ea5 bytes) 2024-09-13T09:27:42.818 INFO:teuthology.orchestra.run.smithi063.stdout:4915 trunc from 0xec24ff7 to 0x33627ec 2024-09-13T09:27:42.835 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:42.831+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:42.851 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:42.868 INFO:teuthology.orchestra.run.smithi063.stdout:4918 trunc from 0x33627ec to 0xe1f173e 2024-09-13T09:27:42.890 INFO:teuthology.orchestra.run.smithi063.stdout:4919 writesame 0xea6834d thru 0xea68dc2 data_size 0xd(0xa76 bytes) 2024-09-13T09:27:42.890 INFO:teuthology.orchestra.run.smithi063.stdout:4920 write 0xb286de1 thru 0xb28a4a3 (0x36c3 bytes) 2024-09-13T09:27:42.891 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:42.891 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:42.891 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:42.891 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:42.891 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:42.901 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:42.902 INFO:teuthology.orchestra.run.smithi063.stdout:4921 read 0x85fff21 thru 0x860da03 (0xdae3 bytes) 2024-09-13T09:27:42.902 INFO:teuthology.orchestra.run.smithi063.stdout:4922 writesame 0x6305c5b thru 0x6306c3d data_size 0x7(0xfe3 bytes) 2024-09-13T09:27:42.910 INFO:teuthology.orchestra.run.smithi063.stdout:4923 writesame 0x473881f thru 0x473f441 data_size 0x1f(0x6c23 bytes) 2024-09-13T09:27:42.924 INFO:teuthology.orchestra.run.smithi063.stdout:4925 read 0x4050dcf thru 0x4056bc4 (0x5df6 bytes) 2024-09-13T09:27:42.924 INFO:teuthology.orchestra.run.smithi063.stdout:4926 punch from 0x36c211b to 0x36c5812, (0x36f7 bytes) 2024-09-13T09:27:42.925 INFO:teuthology.orchestra.run.smithi063.stdout:4929 clone 37 order 18 su 131072 sc 14 2024-09-13T09:27:42.936 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:42.932+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:43.007 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:43.007 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:43.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:43.007 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:43.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.264412565999997, 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:43.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.009 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:43.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:43.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:43.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:43.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124700 3.1e 3:78e395d8:::rbd_data.1118f3eff690.0000000000000080:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:43.019 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.699769836999998, 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011658599, 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124700 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:43.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302296+0000", 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302294+0000", 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302307+0000", 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3602e-05 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302336+0000", 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8116999999999999e-05 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302348+0000", 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2792000000000001e-05 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302394+0000", 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5797000000000002e-05 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.302505+0000", 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111286 2024-09-13T09:27:43.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.304334+0000", 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001828325 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313641+0000", 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093073449999999999 2024-09-13T09:27:43.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313729+0000", 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8305000000000002e-05 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313834+0000", 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010443899999999999 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:06.313955+0000", 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000121142 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128961 3.b 3:d06f0007:::rbd_data.11181500dc3e.0000000000000009:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.026 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.154799616999998, 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010200722000000001, 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128961 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847267+0000", 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847265+0000", 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847762+0000", 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00049738799999999998 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847787+0000", 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4451999999999999e-05 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847799+0000", 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2140999999999999e-05 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847886+0000", 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7020000000000004e-05 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.848076+0000", 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019032700000000001 2024-09-13T09:27:43.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852181+0000", 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041049830000000004 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857348+0000", 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051670809999999996 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857367+0000", 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8533e-05 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857428+0000", 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1334999999999995e-05 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.857467+0000", 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9286000000000003e-05 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129048 3.b 3:d1c06694:::rbd_data.11181500dc3e.0000000000000088:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.032 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.959763123999998, 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014253025000000001, 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129048 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042303+0000", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042302+0000", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042309+0000", 2024-09-13T09:27:43.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.601e-06 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042322+0000", 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3529999999999999e-05 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042327+0000", 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042399+0000", 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1208e-05 2024-09-13T09:27:43.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.042524+0000", 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012532799999999999 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 0", 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.044815+0000", 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022907729999999999 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056394+0000", 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011579271 2024-09-13T09:27:43.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056445+0000", 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0994e-05 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056500+0000", 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5442999999999998e-05 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.056556+0000", 2024-09-13T09:27:43.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5702000000000003e-05 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130752 3.2 3:45d3584e:::rbd_data.1118de6ae5af.000000000000004e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.988479+0000", 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.013586732, 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012273492, 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130752 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988479+0000", 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988479+0000", 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988477+0000", 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988494+0000", 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6617e-05 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988526+0000", 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2660999999999998e-05 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988540+0000", 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3994e-05 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988627+0000", 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6427000000000004e-05 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.988807+0000", 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018076499999999999 2024-09-13T09:27:43.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.990749+0000", 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019417600000000001 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000452+0000", 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0097028979999999997 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000488+0000", 2024-09-13T09:27:43.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5644e-05 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000635+0000", 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014702199999999999 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.000753+0000", 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118272 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130776 3.8 3:1727d39e:::rbd_data.1118de6ae5af.00000000000000c4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.071035+0000", 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.931031004999999, 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015911854999999999, 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130776 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071035+0000", 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071035+0000", 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071032+0000", 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071049+0000", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.702e-05 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071082+0000", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3043999999999997e-05 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071098+0000", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5339999999999999e-05 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071183+0000", 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5415999999999997e-05 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.071389+0000", 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000206269 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.073943+0000", 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025537950000000002 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086726+0000", 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012782401000000001 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086802+0000", 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086885+0000", 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3357999999999997e-05 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.086947+0000", 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2147e-05 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:132654 3.18 e101/75 ondisk, result = 0)", 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:32.880970+0000", 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.121095874, 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010354965000000001, 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880970+0000", 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880970+0000", 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880969+0000", 2024-09-13T09:27:43.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880977+0000", 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5859999999999999e-06 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880989+0000", 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2296000000000001e-05 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.880997+0000", 2024-09-13T09:27:43.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2970000000000004e-06 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.891204+0000", 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010207188000000001 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.891213+0000", 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6219999999999998e-06 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:32.891325+0000", 2024-09-13T09:27:43.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112281 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135402 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.433178+0000", 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.56888813599999999, 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091453369999999999, 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135402 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.433178+0000", 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.433178+0000", 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.433177+0000", 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.433185+0000", 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7980000000000006e-06 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.433200+0000", 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5534000000000001e-05 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.433223+0000", 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.23e-05 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.439093+0000", 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005869834 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.439338+0000", 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024519800000000002 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.440129+0000", 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00079173100000000005 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.442118+0000", 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019881310000000002 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.442184+0000", 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6316999999999995e-05 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.442270+0000", 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6151999999999998e-05 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.442323+0000", 2024-09-13T09:27:43.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3310999999999998e-05 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135440 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.508766+0000", 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.49330033000000001, 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010533684, 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135440 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508766+0000", 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508766+0000", 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508765+0000", 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508770+0000", 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2000000000000002e-06 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508797+0000", 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.635e-05 2024-09-13T09:27:43.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508808+0000", 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1623999999999999e-05 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.512464+0000", 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003655476 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.512618+0000", 2024-09-13T09:27:43.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015466199999999999 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.513418+0000", 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00079913100000000002 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.514983+0000", 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015651630000000001 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519004+0000", 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040212319999999996 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519182+0000", 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017806699999999999 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519300+0000", 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011757299999999999 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.064 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135441 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.508816+0000", 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.49325035299999997, 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010750276, 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.065 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135441 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508816+0000", 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508816+0000", 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508815+0000", 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508820+0000", 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7299999999999996e-06 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508840+0000", 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9910000000000001e-05 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.508847+0000", 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2099999999999996e-06 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.515065+0000", 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.006217752 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.515247+0000", 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018194700000000001 2024-09-13T09:27:43.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.515937+0000", 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000689947 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519339+0000", 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034026479999999999 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519396+0000", 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6501e-05 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519518+0000", 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000122156 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.519566+0000", 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8142000000000001e-05 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135488 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.602042+0000", 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.40002388700000002, 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091839079999999993, 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135488 2024-09-13T09:27:43.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.602042+0000", 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.602042+0000", 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.602042+0000", 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.602047+0000", 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1120000000000004e-06 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.602072+0000", 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5111999999999999e-05 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.602088+0000", 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5767999999999999e-05 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.607758+0000", 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056709029999999997 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.607925+0000", 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000166966 2024-09-13T09:27:43.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.608508+0000", 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00058228100000000003 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.609932+0000", 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001423936 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.611081+0000", 2024-09-13T09:27:43.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011492609999999999 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.611168+0000", 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6827000000000001e-05 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.611226+0000", 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8523999999999997e-05 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135518 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.666633+0000", 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.33543359299999997, 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011606864999999999, 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135518 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666633+0000", 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666633+0000", 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666631+0000", 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666640+0000", 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2470000000000008e-06 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666655+0000", 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5713e-05 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666664+0000", 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9870000000000005e-06 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.669768+0000", 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003103494 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.669894+0000", 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012667700000000001 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.670531+0000", 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00063631199999999999 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.676352+0000", 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0058213020000000004 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678153+0000", 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001801247 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678203+0000", 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9178000000000002e-05 2024-09-13T09:27:43.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678239+0000", 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6937999999999998e-05 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135519 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.666671+0000", 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.335395149, 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011901544999999999, 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135519 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666671+0000", 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666671+0000", 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666670+0000", 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666675+0000", 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666686+0000", 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0923e-05 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666692+0000", 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3730000000000001e-06 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.671737+0000", 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050454419999999998 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.671883+0000", 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014587000000000001 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.672399+0000", 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00051581100000000003 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.676359+0000", 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039598869999999996 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678408+0000", 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020490059999999999 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678469+0000", 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1904999999999998e-05 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678573+0000", 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103088 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135520 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.666719+0000", 2024-09-13T09:27:43.087 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.33534688299999998, 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012017076, 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135520 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666719+0000", 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666719+0000", 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666719+0000", 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666724+0000", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5670000000000002e-06 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666733+0000", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859999999999995e-06 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666740+0000", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7669999999999998e-06 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.673049+0000", 2024-09-13T09:27:43.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0063095499999999997 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.673148+0000", 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9149999999999998e-05 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.673566+0000", 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000417544 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678294+0000", 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047280320000000001 2024-09-13T09:27:43.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678634+0000", 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033990399999999998 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678701+0000", 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7578999999999998e-05 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678736+0000", 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5070000000000001e-05 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135521 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.666748+0000", 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.33531786099999999, 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.01231203, 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135521 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666748+0000", 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666748+0000", 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666748+0000", 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666752+0000", 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8970000000000001e-06 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666758+0000", 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1430000000000003e-06 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666763+0000", 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9429999999999999e-06 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.674872+0000", 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0081092279999999996 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.675069+0000", 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019675000000000001 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.675594+0000", 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00052487900000000001 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678754+0000", 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031599010000000001 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.678879+0000", 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000125437 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.679000+0000", 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120701 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.679060+0000", 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0646000000000002e-05 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135522 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.666767+0000", 2024-09-13T09:27:43.098 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.335299079, 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014430912000000001, 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135522 2024-09-13T09:27:43.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666767+0000", 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666767+0000", 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666767+0000", 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666771+0000", 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.89e-06 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666777+0000", 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0870000000000003e-06 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666782+0000", 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2660000000000001e-06 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.676515+0000", 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0097330160000000006 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.676675+0000", 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000160178 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.677295+0000", 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00062034299999999998 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.680944+0000", 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0036486779999999998 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681037+0000", 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2462999999999999e-05 2024-09-13T09:27:43.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681140+0000", 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010359199999999999 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681198+0000", 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7886000000000001e-05 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135523 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.666786+0000", 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.33527993900000003, 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015198781, 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135523 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.105 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666786+0000", 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666786+0000", 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666786+0000", 2024-09-13T09:27:43.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666790+0000", 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7400000000000002e-06 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666796+0000", 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1979999999999998e-06 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.666801+0000", 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3569999999999997e-06 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.679160+0000", 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.012358757999999999 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.679328+0000", 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016851100000000001 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.680076+0000", 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00074789699999999997 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681516+0000", 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0014398939999999999 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681882+0000", 2024-09-13T09:27:43.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036583999999999998 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681941+0000", 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9398999999999998e-05 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.681985+0000", 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3643999999999998e-05 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135565 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.731622+0000", 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.27044380600000001, 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0097324750000000008, 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135565 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.111 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.731622+0000", 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.731622+0000", 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.731621+0000", 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.731630+0000", 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3799999999999994e-06 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.731648+0000", 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8564999999999999e-05 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.731661+0000", 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3322999999999999e-05 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.734433+0000", 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027719910000000001 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.734627+0000", 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019378 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.735178+0000", 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00055085900000000001 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.736149+0000", 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00097069300000000001 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.741250+0000", 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051017279999999998 2024-09-13T09:27:43.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.741300+0000", 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9722000000000003e-05 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.741355+0000", 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4691999999999997e-05 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.116 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135719 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.910201+0000", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.091865031999999999, 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090755609999999994, 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135719 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910201+0000", 2024-09-13T09:27:43.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910201+0000", 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910201+0000", 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910205+0000", 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910210+0000", 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3419999999999998e-06 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910221+0000", 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.064e-05 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.913057+0000", 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0028364179999999998 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.913164+0000", 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106772 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.913599+0000", 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00043478499999999998 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.916257+0000", 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026580190000000002 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.919214+0000", 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029576749999999999 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.919249+0000", 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5145999999999999e-05 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.919277+0000", 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7263999999999998e-05 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135720 3.7 3:e5d05949:::rbd_object_map.11183e1674b5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.910230+0000", 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.091835743999999997, 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0093777670000000004, 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135720 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910230+0000", 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910230+0000", 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910230+0000", 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910233+0000", 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5839999999999999e-06 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910242+0000", 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.910246+0000", 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6140000000000002e-06 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.914250+0000", 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040040120000000004 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.914366+0000", 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116156 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.914978+0000", 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000611799 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.916323+0000", 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013445499999999999 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.919469+0000", 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003146336 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.919547+0000", 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7733999999999998e-05 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.919608+0000", 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1415999999999996e-05 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.127 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:43.129 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:43.129 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:43.124+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:43.183 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:27:43.230 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:43.224+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:43.289 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:43.289 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:43.299 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:43.331 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:43.327+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:43.398 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:43.399 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:43.399 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:43.399 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:43.399 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:43.409 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:43.432 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:43.428+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:43.469 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:43.469 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:43.469 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T09:27:43.470 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T09:27:43.521 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:43.521 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:43.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:43.521 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:43.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.521 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.955081489999998, 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:43.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:43.526 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 53.355857397000001, 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.527 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:43.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 53.247512208000003, 2024-09-13T09:27:43.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:43.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:43.539 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.709167254, 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.709008498999999, 2024-09-13T09:27:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:43.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:43.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:43.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:43.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.708932558999997, 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:43.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:43.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.560 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.707953115999999, 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:43.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.700374396000001, 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.565 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:43.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.433628431999999, 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.571 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:43.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.485557401000001, 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114301, 2024-09-13T09:27:43.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031615+0000", 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031627+0000", 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2072999999999999e-05 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031658+0000", 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1037000000000001e-05 2024-09-13T09:27:43.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031679+0000", 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0557e-05 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040706+0000", 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090265759999999997 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040717+0000", 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1357e-05 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040731+0000", 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4355000000000001e-05 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:43.581 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.485410528999999, 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114278, 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:43.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031763+0000", 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031769+0000", 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4659999999999996e-06 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031799+0000", 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9164000000000001e-05 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031812+0000", 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3465999999999999e-05 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040746+0000", 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0089339659999999998 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040752+0000", 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7830000000000004e-06 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040878+0000", 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012623300000000001 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.367423111000001, 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128541 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.590 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.250884798000001, 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0085633649999999999, 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128541 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266290+0000", 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266289+0000", 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266293+0000", 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0759999999999996e-06 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266301+0000", 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4839999999999998e-06 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.266306+0000", 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5860000000000004e-06 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270348+0000", 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040418859999999997 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270447+0000", 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8554000000000005e-05 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:43.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.270812+0000", 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000364932 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.273865+0000", 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030535940000000002 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274619+0000", 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00075377899999999999 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274786+0000", 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000167298 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.274853+0000", 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6674000000000006e-05 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.596 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128959 3.1f 3:fd5f8e4a:::rbd_data.11181500dc3e.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.681438522000001, 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018621092999999998, 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128959 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835734+0000", 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835742+0000", 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1089999999999998e-06 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835759+0000", 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6591000000000001e-05 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835766+0000", 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835845+0000", 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9524000000000006e-05 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835977+0000", 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013168399999999999 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.839738+0000", 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003761398 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854287+0000", 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014548442999999999 2024-09-13T09:27:43.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854295+0000", 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5690000000000005e-06 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854329+0000", 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4198e-05 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854357+0000", 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7491000000000002e-05 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129111 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.390593837000001, 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086272250000000005, 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.603 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129111 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:43.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126580+0000", 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:43.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126586+0000", 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.178e-06 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126599+0000", 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3124e-05 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126609+0000", 2024-09-13T09:27:43.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0294e-05 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.130890+0000", 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042807009999999996 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131001+0000", 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110368 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:43.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131416+0000", 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00041545800000000001 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134852+0000", 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034356009999999999 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135120+0000", 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026872400000000002 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135173+0000", 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3149000000000002e-05 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135208+0000", 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4297e-05 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129312 3.1 3:875c0977:::rbd_data.11181500dc3e.00000000000000ea:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.609 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.977661958999999, 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086345329999999998, 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129312 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:43.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539511+0000", 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539520+0000", 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0629999999999995e-06 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539535+0000", 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5242000000000001e-05 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539546+0000", 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1185999999999999e-05 2024-09-13T09:27:43.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539603+0000", 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7358000000000002e-05 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539688+0000", 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4495999999999999e-05 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:43.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.541358+0000", 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016698030000000001 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548023+0000", 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066649250000000004 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548069+0000", 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6715000000000003e-05 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548116+0000", 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6226000000000001e-05 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548147+0000", 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1301999999999999e-05 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129371 3.1f 3:fe73dde2:::rbd_data.11181500dc3e.000000000000000f:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.875319065999999, 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014271700999999999, 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129371 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641854+0000", 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641862+0000", 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641877+0000", 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5327000000000001e-05 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641886+0000", 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4129999999999995e-06 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641935+0000", 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8775e-05 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.642014+0000", 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8574999999999999e-05 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:43.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.644683+0000", 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026697359999999998 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655832+0000", 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011148371000000001 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655974+0000", 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014196400000000001 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656079+0000", 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010503800000000001 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656127+0000", 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.812e-05 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.621 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130769 3.1 3:84b2050c:::rbd_data.1118de6ae5af.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.482069363000001, 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024401849, 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130769 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:43.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035103+0000", 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035114+0000", 2024-09-13T09:27:43.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1613e-05 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035136+0000", 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1472e-05 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035145+0000", 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4860000000000003e-06 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035204+0000", 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9027000000000003e-05 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035363+0000", 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015868600000000001 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.040402+0000", 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050390000000000001 2024-09-13T09:27:43.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059321+0000", 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018919062 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059340+0000", 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9001999999999999e-05 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059432+0000", 2024-09-13T09:27:43.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1535999999999999e-05 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059507+0000", 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5220000000000001e-05 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130832 3.12 3:4988a482:::rbd_data.1118de6ae5af.000000000000005e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:43.627 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.298460786, 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090196749999999996, 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130832 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218710+0000", 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218726+0000", 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6439999999999998e-05 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218794+0000", 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7726999999999997e-05 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218815+0000", 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1449000000000001e-05 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218935+0000", 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011945199999999999 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219123+0000", 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000187966 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219363+0000", 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024062399999999999 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.222351+0000", 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029879989999999999 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227642+0000", 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052903109999999998 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227693+0000", 2024-09-13T09:27:43.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1786000000000002e-05 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227733+0000", 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9761000000000001e-05 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135322 3.1b 3:d9011f0c:::rbd_data.11183e1674b5.000000000000009f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.328576+0000", 2024-09-13T09:27:43.633 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.1885983630000001, 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090168950000000005, 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135322 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:43.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328576+0000", 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328576+0000", 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328575+0000", 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:43.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328584+0000", 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0799999999999995e-06 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328606+0000", 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2555e-05 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328619+0000", 2024-09-13T09:27:43.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.294e-05 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328719+0000", 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010016 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328800+0000", 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0822999999999995e-05 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:43.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328869+0000", 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8977e-05 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.329639+0000", 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077026199999999996 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.337467+0000", 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078272250000000002 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.337545+0000", 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8376999999999999e-05 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.337593+0000", 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7888e-05 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:43.639 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:43.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:43.641 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone36 from 0xea68dc3 (overlap 0x288a7b) to 0x14453d 2024-09-13T09:27:43.642 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:43.635+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:43.742 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:43.736+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:43.753 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T09:27:43.763 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:43.763 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:43.763 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:43.763 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:43.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:43.787 INFO:teuthology.orchestra.run.smithi063.stdout:4932 read 0x489c32e thru 0x48a791a (0xb5ed bytes) 2024-09-13T09:27:43.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:27:43.784+0000 7f5baa59c700 0 --2- 172.21.15.63:0/3595368133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bac0663c0 0x7f5bac13a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:27:43.793 INFO:teuthology.orchestra.run.smithi063.stdout:4933 write 0x68a2d03 thru 0x68aa292 (0x7590 bytes) 2024-09-13T09:27:43.800 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:43.800 INFO:teuthology.orchestra.run.smithi063.stdout:4935 writesame 0xe5f146a thru 0xe5fae20 data_size 0xd(0x99b7 bytes) 2024-09-13T09:27:43.806 INFO:teuthology.orchestra.run.smithi063.stdout:4936 read 0x654ac27 thru 0x6553bdc (0x8fb6 bytes) 2024-09-13T09:27:43.806 INFO:teuthology.orchestra.run.smithi063.stdout:4937 read 0xba13ce9 thru 0xba1a6c6 (0x69de bytes) 2024-09-13T09:27:43.807 INFO:teuthology.orchestra.run.smithi063.stdout:4938 punch from 0xdc00c1f to 0xdc02e25, (0x2206 bytes) 2024-09-13T09:27:43.807 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:43.807 INFO:teuthology.orchestra.run.smithi063.stdout:4940 writesame 0xd172934 thru 0xd181c80 data_size 0x1(0xf34d bytes) 2024-09-13T09:27:43.843 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:43.836+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:43.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T09:27:43.851 INFO:teuthology.orchestra.run.smithi063.stdout:4941 trunc from 0xea68dc3 to 0xdc4a477 2024-09-13T09:27:43.882 INFO:teuthology.orchestra.run.smithi063.stdout:4942 write 0x5a6a0ac thru 0x5a6f557 (0x54ac bytes) 2024-09-13T09:27:43.892 INFO:teuthology.orchestra.run.smithi063.stdout:4943 trunc from 0xdc4a477 to 0xcdfc459 2024-09-13T09:27:43.898 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:43.898 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:43.898 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:43.898 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:43.898 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:43.910 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:43.930 INFO:teuthology.orchestra.run.smithi063.stdout:4944 write 0x8e81737 thru 0x8e83d0f (0x25d9 bytes) 2024-09-13T09:27:43.937 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:43.937 INFO:teuthology.orchestra.run.smithi063.stdout:4946 write 0x13eddd8 thru 0x13f76d7 (0x9900 bytes) 2024-09-13T09:27:43.944 INFO:teuthology.orchestra.run.smithi063.stdout:4947 punch from 0x40d765d to 0x40dd6bc, (0x605f bytes) 2024-09-13T09:27:43.944 INFO:teuthology.orchestra.run.smithi063.stdout:4949 read 0x72a5310 thru 0x72b0ebb (0xbbac bytes) 2024-09-13T09:27:43.944 INFO:teuthology.orchestra.run.smithi063.stdout:4952 punch from 0x7fa06f5 to 0x7fa83e6, (0x7cf1 bytes) 2024-09-13T09:27:43.944 INFO:teuthology.orchestra.run.smithi063.stdout:4953 write 0x7985c41 thru 0x798c9d5 (0x6d95 bytes) 2024-09-13T09:27:43.944 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:43.940+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:43.952 INFO:teuthology.orchestra.run.smithi063.stdout:4954 write 0x4536a8f thru 0x45454c4 (0xea36 bytes) 2024-09-13T09:27:43.960 INFO:teuthology.orchestra.run.smithi063.stdout:4955 write 0xa43838a thru 0xa446b69 (0xe7e0 bytes) 2024-09-13T09:27:43.968 INFO:teuthology.orchestra.run.smithi063.stdout:4956 writesame 0xae17d72 thru 0xae22005 data_size 0xa(0xa294 bytes) 2024-09-13T09:27:43.976 INFO:teuthology.orchestra.run.smithi063.stdout:4957 punch from 0x1282b0e to 0x1290802, (0xdcf4 bytes) 2024-09-13T09:27:43.976 INFO:teuthology.orchestra.run.smithi063.stdout:4959 read 0x8b83f93 thru 0x8b8653e (0x25ac bytes) 2024-09-13T09:27:43.976 INFO:teuthology.orchestra.run.smithi063.stdout:4961 read 0x62197b2 thru 0x6225f71 (0xc7c0 bytes) 2024-09-13T09:27:43.978 INFO:teuthology.orchestra.run.smithi063.stdout:4962 write 0x5e3443c thru 0x5e359ce (0x1593 bytes) 2024-09-13T09:27:43.985 INFO:teuthology.orchestra.run.smithi063.stdout:4963 flatten 2024-09-13T09:27:44.041 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T09:27:44.045 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:44.039+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:44.059 INFO:teuthology.orchestra.run.smithi063.stdout:4966 writesame 0xd195e13 thru 0xd1a2a82 data_size 0x8(0xcc70 bytes) 2024-09-13T09:27:44.060 INFO:teuthology.orchestra.run.smithi063.stdout:4967 write 0xcba52b2 thru 0xcbb3c05 (0xe954 bytes) 2024-09-13T09:27:44.065 INFO:teuthology.orchestra.run.smithi063.stdout:4969 clone 38 order 19 su 32768 sc 5 2024-09-13T09:27:44.070 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:44.070 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 94.113107735, 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:44.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:44.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:44.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.420574533000007, 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.077 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:44.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.947801350000006, 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517448354000003, 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.51742248, 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517397115999998, 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517312093999998, 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517278699000002, 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517264761, 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517227015000003, 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517209098999999, 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517195450999999, 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517128806000002, 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517080745999998, 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517023473999998, 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.517003365000001, 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.516981219000002, 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.516924951999997, 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.516909056000003, 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:136073 3.0 3:046f580d:::rbd_data.1118d1656b7c.0000000000000343:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.25550551300000002, 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.038067780000000002, 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 136073 2024-09-13T09:27:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811437+0000", 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1569999999999997e-06 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811448+0000", 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1297999999999999e-05 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811453+0000", 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4539999999999998e-06 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811531+0000", 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7268000000000006e-05 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811730+0000", 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019893499999999999 2024-09-13T09:27:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.819515+0000", 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0077851379999999996 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849352+0000", 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029836813 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849368+0000", 2024-09-13T09:27:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6218e-05 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849423+0000", 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5417000000000002e-05 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849499+0000", 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6029e-05 2024-09-13T09:27:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.166 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:44.168 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:44.168 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:44.160+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:44.222 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T09:27:44.268 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:44.263+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:44.317 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:44.317 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:44.326 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:44.369 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:44.367+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:44.385 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:27:44.392 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:44.394 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T09:27:44.433 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:44.433 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:44.433 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:44.433 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:44.433 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:44.444 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:44.471 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:44.467+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:44.545 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:44.560 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-13T09:27:44.572 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:44.568+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:44.581 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:44.602 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:3063 3.9 e31/21 3:951c3f22:::rbd_data.11189888a71f.0000000000000009:head v 31'29, mlcod=31'27)", 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 99.198136281999993, 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041086930000000001, 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401024+0000", 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.401023+0000", 2024-09-13T09:27:44.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403412+0000", 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023893679999999998 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403428+0000", 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5696e-05 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403438+0000", 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0030999999999999e-05 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403571+0000", 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013305000000000001 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.403616+0000", 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4727000000000001e-05 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405080+0000", 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001464695 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:05.405132+0000", 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2064999999999997e-05 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 94.633026896999993, 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:44.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 94.559487884000006, 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.612 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:44.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 80.438883939999997, 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:44.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 80.424626157000006, 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:44.621 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:44.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.966983010999996, 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:44.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.761956963000003, 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:44.630 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:44.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.588217739999997, 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:44.635 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:44.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.394135296999998, 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.640 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:44.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:44.644 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 57.009772456, 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.649 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:128958 3.13 e95/94 3:ca28514f:::rbd_data.11181500dc3e.000000000000000a:head v 95'531, mlcod=95'529)", 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.751190057999999, 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042980650000000002, 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847968+0000", 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850296+0000", 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002328044 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850322+0000", 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5741e-05 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850334+0000", 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1936999999999999e-05 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850436+0000", 2024-09-13T09:27:44.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000102631 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850488+0000", 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2169000000000001e-05 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852145+0000", 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016569480000000001 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852268+0000", 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012255999999999999 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128966 3.9 3:9420bb05:::rbd_data.11181500dc3e.000000000000008f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.710409126999998, 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053551420000000002, 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.654 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128966 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:44.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888748+0000", 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888764+0000", 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6772000000000001e-05 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888807+0000", 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2827000000000002e-05 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888829+0000", 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1426000000000001e-05 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888965+0000", 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013610599999999999 2024-09-13T09:27:44.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889174+0000", 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020964700000000001 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889456+0000", 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000281403 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.890649+0000", 2024-09-13T09:27:44.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011928819999999999 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893917+0000", 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003268328 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894020+0000", 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103003 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894106+0000", 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5986e-05 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130683 3.9 3:9160b546:::rbd_data.1118de6ae5af.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.788438933, 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005777351, 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.660 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130683 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810718+0000", 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811313+0000", 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059520999999999999 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.662 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811352+0000", 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9453999999999999e-05 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811369+0000", 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6886999999999999e-05 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811475+0000", 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105845 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811814+0000", 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033890599999999998 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812483+0000", 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00066937099999999999 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:44.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816386+0000", 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039026949999999999 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816411+0000", 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4916e-05 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816465+0000", 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3835000000000002e-05 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816498+0000", 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3555e-05 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:133033 3.13 e101/94 3:c9a00536:::rbd_data.11182b78662d.000000000000001a:head v 101'625, mlcod=101'623)", 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 11.44445571, 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042112800000000004, 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.666 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.667 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154702+0000", 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156468+0000", 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017665440000000001 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156503+0000", 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:27:44.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156518+0000", 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5543e-05 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156666+0000", 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014814599999999999 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156733+0000", 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7037000000000002e-05 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.158851+0000", 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021177840000000002 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.158915+0000", 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4343000000000004e-05 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.670 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136071 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'381, mlcod=111'376)", 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.78886152499999995, 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048782419999999996, 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:44.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.672 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810306+0000", 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3920000000000003e-06 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810322+0000", 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810331+0000", 2024-09-13T09:27:44.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4160000000000004e-06 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810388+0000", 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7163999999999997e-05 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810414+0000", 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6163999999999999e-05 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815117+0000", 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047025620000000004 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815177+0000", 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9892999999999997e-05 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.675 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136080 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'385, mlcod=111'382)", 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.73662720999999998, 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057524539999999997, 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.676 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862531+0000", 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.677 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862543+0000", 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1538e-05 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862560+0000", 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7156e-05 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862572+0000", 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1606e-05 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.678 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862640+0000", 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8695999999999994e-05 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862669+0000", 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8354999999999999e-05 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868211+0000", 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055418450000000001 2024-09-13T09:27:44.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868285+0000", 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4486000000000005e-05 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136101 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'391, mlcod=111'388)", 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.69338679400000003, 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050269909999999998, 2024-09-13T09:27:44.680 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.681 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905772+0000", 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905783+0000", 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0822000000000001e-05 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905796+0000", 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3328e-05 2024-09-13T09:27:44.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905805+0000", 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6079999999999996e-06 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905898+0000", 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2958e-05 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905922+0000", 2024-09-13T09:27:44.683 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4572000000000001e-05 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910718+0000", 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047962489999999998 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910800+0000", 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1749999999999995e-05 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136103 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'393, mlcod=111'388)", 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.69101798299999995, 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052766970000000003, 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.685 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908141+0000", 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908153+0000", 2024-09-13T09:27:44.686 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2686e-05 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908170+0000", 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6651999999999999e-05 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908187+0000", 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908236+0000", 2024-09-13T09:27:44.687 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8959000000000003e-05 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908275+0000", 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8983999999999998e-05 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913370+0000", 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050951190000000004 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913419+0000", 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8293e-05 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136104 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'394, mlcod=111'388)", 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.909366+0000", 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.68979354999999998, 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0041010059999999999, 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.689 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909366+0000", 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909366+0000", 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909366+0000", 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909374+0000", 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0590000000000003e-06 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909389+0000", 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5410999999999999e-05 2024-09-13T09:27:44.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909396+0000", 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1609999999999997e-06 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909442+0000", 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5516000000000003e-05 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.909454+0000", 2024-09-13T09:27:44.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1932e-05 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913423+0000", 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039692909999999998 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913467+0000", 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4480000000000001e-05 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.693 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136328 3.9 e111/107 3:934af333:::rbd_data.1118d1656b7c.000000000000000b:head v 111'255, mlcod=110'253)", 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.59012384600000001, 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044988249999999997, 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:44.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009035+0000", 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:44.695 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009276+0000", 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024107299999999999 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009292+0000", 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6396e-05 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009300+0000", 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6559999999999992e-06 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.696 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009375+0000", 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4899000000000002e-05 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009402+0000", 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7404999999999999e-05 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.013491+0000", 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040891260000000002 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:44.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.013535+0000", 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3401e-05 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:44.698 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:44.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:44.700 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:44.701 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:44.701 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:44.701 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:44.701 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:44.695+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:44.717 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T09:27:44.725 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:44.731 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T09:27:44.751 INFO:teuthology.orchestra.run.smithi063.stdout:4970 read 0x5d59aff thru 0x5d5c306 (0x2808 bytes) 2024-09-13T09:27:44.752 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:44.755 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:44.758 INFO:teuthology.orchestra.run.smithi063.stdout:4971 writesame 0x6e70e4e thru 0x6e7d1b8 data_size 0x1(0xc36b bytes) 2024-09-13T09:27:44.784 INFO:teuthology.orchestra.run.smithi063.stdout:4972 read 0xaf15f8d thru 0xaf19843 (0x38b7 bytes) 2024-09-13T09:27:44.784 INFO:teuthology.orchestra.run.smithi063.stdout:4973 write 0x6f6bc2 thru 0x6f955e (0x299d bytes) 2024-09-13T09:27:44.794 INFO:teuthology.orchestra.run.smithi063.stdout:4974 read 0x5f05f62 thru 0x5f13c4c (0xdceb bytes) 2024-09-13T09:27:44.800 INFO:teuthology.orchestra.run.smithi063.stdout:4976 trunc from 0xd1a2a83 to 0xa072bc5 2024-09-13T09:27:44.800 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:44.792+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:44.820 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:44.821 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:44.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:44.841 INFO:teuthology.orchestra.run.smithi063.stdout:4979 write 0xa422566 thru 0xa42eead (0xc948 bytes) 2024-09-13T09:27:44.858 INFO:teuthology.orchestra.run.smithi063.stdout:4980 write 0xa437962 thru 0xa43c761 (0x4e00 bytes) 2024-09-13T09:27:44.866 INFO:teuthology.orchestra.run.smithi063.stdout:4981 punch from 0x86574a3 to 0x865c6de, (0x523b bytes) 2024-09-13T09:27:44.866 INFO:teuthology.orchestra.run.smithi063.stdout:4982 read 0x5e6bb4c thru 0x5e6d411 (0x18c6 bytes) 2024-09-13T09:27:44.866 INFO:teuthology.orchestra.run.smithi063.stdout:4983 read 0x22833f5 thru 0x229177a (0xe386 bytes) 2024-09-13T09:27:44.866 INFO:teuthology.orchestra.run.smithi063.stdout:4984 read 0x1e7287c thru 0x1e75eb4 (0x3639 bytes) 2024-09-13T09:27:44.866 INFO:teuthology.orchestra.run.smithi063.stdout:4985 trunc from 0xa43c762 to 0xa391d7a 2024-09-13T09:27:44.887 INFO:teuthology.orchestra.run.smithi063.stdout:4987 write 0x83e2e thru 0x8be82 (0x8055 bytes) 2024-09-13T09:27:44.900 INFO:teuthology.orchestra.run.smithi063.stdout:4988 read 0x5b87e68 thru 0x5b95e7c (0xe015 bytes) 2024-09-13T09:27:44.900 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:44.895+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:44.908 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T09:27:44.916 INFO:teuthology.orchestra.run.smithi063.stdout:4991 write 0xb5efd7f thru 0xb5f071a (0x99c bytes) 2024-09-13T09:27:44.922 INFO:teuthology.orchestra.run.smithi063.stdout:4992 read 0x531db5 thru 0x5400ab (0xe2f7 bytes) 2024-09-13T09:27:44.923 INFO:teuthology.orchestra.run.smithi063.stdout:4996 read 0x4b37de thru 0x4b3ea0 (0x6c3 bytes) 2024-09-13T09:27:44.939 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:44.939 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:44.944 INFO:teuthology.orchestra.run.smithi063.stdout:4997 write 0xecbfec3 thru 0xecc9abc (0x9bfa bytes) 2024-09-13T09:27:44.949 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:44.953 INFO:teuthology.orchestra.run.smithi063.stdout:5000 write 0x9a46977 thru 0x9a49dbb (0x3445 bytes) 2024-09-13T09:27:44.960 INFO:teuthology.orchestra.run.smithi063.stdout:5001 read 0xbfc16c9 thru 0xbfca3d6 (0x8d0e bytes) 2024-09-13T09:27:44.960 INFO:teuthology.orchestra.run.smithi063.stdout:5002 punch from 0xda7065a to 0xda76f49, (0x68ef bytes) 2024-09-13T09:27:44.960 INFO:teuthology.orchestra.run.smithi063.stdout:5003 read 0x114bffd thru 0x1158ee8 (0xceec bytes) 2024-09-13T09:27:44.960 INFO:teuthology.orchestra.run.smithi063.stdout:5004 write 0x10aa1a5 thru 0x10aad9c (0xbf8 bytes) 2024-09-13T09:27:44.970 INFO:teuthology.orchestra.run.smithi063.stdout:5005 read 0x76f1a08 thru 0x76f73dc (0x59d5 bytes) 2024-09-13T09:27:44.970 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:44.970 INFO:teuthology.orchestra.run.smithi063.stdout:5007 writesame 0xe5fc6ce thru 0xe6072c2 data_size 0x1(0xabf5 bytes) 2024-09-13T09:27:44.993 INFO:teuthology.orchestra.run.smithi063.stdout:5008 writesame 0x765c3b8 thru 0x76629fa data_size 0x1(0x6643 bytes) 2024-09-13T09:27:45.001 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:44.996+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:45.010 INFO:teuthology.orchestra.run.smithi063.stdout:5009 write 0xcbc4443 thru 0xcbc4530 (0xee bytes) 2024-09-13T09:27:45.015 INFO:teuthology.orchestra.run.smithi063.stdout:5010 write 0xff2177 thru 0xffd6d0 (0xb55a bytes) 2024-09-13T09:27:45.021 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:45.021 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:45.021 INFO:teuthology.orchestra.run.smithi063.stdout:5014 write 0xccf65dc thru 0xccfff7a (0x999f bytes) 2024-09-13T09:27:45.027 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:45.027 INFO:teuthology.orchestra.run.smithi063.stdout:5017 read 0x16f0462 thru 0x16fecb2 (0xe851 bytes) 2024-09-13T09:27:45.027 INFO:teuthology.orchestra.run.smithi063.stdout:5018 write 0x8790be6 thru 0x87994f8 (0x8913 bytes) 2024-09-13T09:27:45.037 INFO:teuthology.orchestra.run.smithi063.stdout:5021 punch from 0x5f08c9b to 0x5f17490, (0xe7f5 bytes) 2024-09-13T09:27:45.037 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:45.037 INFO:teuthology.orchestra.run.smithi063.stdout:5023 read 0x9f3366e thru 0x9f3e8ae (0xb241 bytes) 2024-09-13T09:27:45.037 INFO:teuthology.orchestra.run.smithi063.stdout:5025 write 0x49ac1a6 thru 0x49b1bb7 (0x5a12 bytes) 2024-09-13T09:27:45.048 INFO:teuthology.orchestra.run.smithi063.stdout:5026 read 0xd98c254 thru 0xd998c7b (0xca28 bytes) 2024-09-13T09:27:45.048 INFO:teuthology.orchestra.run.smithi063.stdout:5027 read 0x78c9c23 thru 0x78c9fcb (0x3a9 bytes) 2024-09-13T09:27:45.048 INFO:teuthology.orchestra.run.smithi063.stdout:5028 read 0x97e363d thru 0x97eea6a (0xb42e bytes) 2024-09-13T09:27:45.049 INFO:teuthology.orchestra.run.smithi063.stdout:5029 write 0x486ab7f thru 0x4875bb9 (0xb03b bytes) 2024-09-13T09:27:45.059 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.060 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:45.088 INFO:teuthology.orchestra.run.smithi063.stdout:5030 trunc from 0xecc9abd to 0x4cd06 2024-09-13T09:27:45.101 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:45.099+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:45.131 INFO:teuthology.orchestra.run.smithi063.stdout:5031 punch from 0x2367a to 0x2f449, (0xbdcf bytes) 2024-09-13T09:27:45.150 INFO:teuthology.orchestra.run.smithi063.stdout:5032 write 0x52fa797 thru 0x52fe08b (0x38f5 bytes) 2024-09-13T09:27:45.155 INFO:teuthology.orchestra.run.smithi063.stdout:5033 write 0x2f6ef2b thru 0x2f756a9 (0x677f bytes) 2024-09-13T09:27:45.161 INFO:teuthology.orchestra.run.smithi063.stdout:5034 write 0x2be3888 thru 0x2be923c (0x59b5 bytes) 2024-09-13T09:27:45.179 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.179 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:45.190 INFO:teuthology.orchestra.run.smithi063.stdout:5036 write 0x9feaa78 thru 0x9fec00c (0x1595 bytes) 2024-09-13T09:27:45.197 INFO:teuthology.orchestra.run.smithi063.stdout:5037 write 0x7ace14a thru 0x7ad26b0 (0x4567 bytes) 2024-09-13T09:27:45.202 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:45.196+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:45.208 INFO:teuthology.orchestra.run.smithi063.stdout:5038 read 0x33ff8aa thru 0x340f7d6 (0xff2d bytes) 2024-09-13T09:27:45.208 INFO:teuthology.orchestra.run.smithi063.stdout:5039 read 0x8d08792 thru 0x8d17146 (0xe9b5 bytes) 2024-09-13T09:27:45.208 INFO:teuthology.orchestra.run.smithi063.stdout:5040 read 0x4df697e thru 0x4e01c50 (0xb2d3 bytes) 2024-09-13T09:27:45.208 INFO:teuthology.orchestra.run.smithi063.stdout:5041 write 0x631de1c thru 0x632561e (0x7803 bytes) 2024-09-13T09:27:45.234 INFO:teuthology.orchestra.run.smithi063.stdout:5042 writesame 0xe3755c5 thru 0xe381736 data_size 0xb(0xc172 bytes) 2024-09-13T09:27:45.234 INFO:teuthology.orchestra.run.smithi063.stdout:5044 punch from 0xba42630 to 0xba432dc, (0xcac bytes) 2024-09-13T09:27:45.242 INFO:teuthology.orchestra.run.smithi063.stdout:5046 trunc from 0xe381737 to 0x9bd1a99 2024-09-13T09:27:45.262 INFO:teuthology.orchestra.run.smithi063.stdout:5047 write 0x5253204 thru 0x525b6b0 (0x84ad bytes) 2024-09-13T09:27:45.269 INFO:teuthology.orchestra.run.smithi063.stdout:5050 read 0x1155cf5 thru 0x115d4cf (0x77db bytes) 2024-09-13T09:27:45.285 INFO:teuthology.orchestra.run.smithi063.stdout:5051 write 0xb8e3a34 thru 0xb8ed756 (0x9d23 bytes) 2024-09-13T09:27:45.293 INFO:teuthology.orchestra.run.smithi063.stdout:5052 write 0x7fe5db3 thru 0x7fea83e (0x4a8c bytes) 2024-09-13T09:27:45.303 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:45.299+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:45.318 INFO:teuthology.orchestra.run.smithi063.stdout:5053 writesame 0xde510f4 thru 0xde5f088 data_size 0x1(0xdf95 bytes) 2024-09-13T09:27:45.338 INFO:teuthology.orchestra.run.smithi063.stdout:5054 trunc from 0xde5f089 to 0x3da9971 2024-09-13T09:27:45.348 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.348 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.358 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:5055 read 0x3835826 thru 0x383a0db (0x48b6 bytes) 2024-09-13T09:27:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:5056 writesame 0x3d2173c thru 0x3d28b5e data_size 0x1(0x7423 bytes) 2024-09-13T09:27:45.381 INFO:teuthology.orchestra.run.smithi063.stdout:5057 read 0x86d97b thru 0x86dbd4 (0x25a bytes) 2024-09-13T09:27:45.381 INFO:teuthology.orchestra.run.smithi063.stdout:5059 read 0x1ada552 thru 0x1ae80fc (0xdbab bytes) 2024-09-13T09:27:45.382 INFO:teuthology.orchestra.run.smithi063.stdout:5060 read 0x13a7dc1 thru 0x13ac26f (0x44af bytes) 2024-09-13T09:27:45.382 INFO:teuthology.orchestra.run.smithi063.stdout:5061 read 0x9424c2 thru 0x94bb9f (0x96de bytes) 2024-09-13T09:27:45.382 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:45.400 INFO:teuthology.orchestra.run.smithi063.stdout:5063 write 0x6ba2376 thru 0x6bacde4 (0xaa6f bytes) 2024-09-13T09:27:45.404 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:45.400+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:45.409 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:45.409 INFO:teuthology.orchestra.run.smithi063.stdout:5065 writesame 0x3b9f46f thru 0x3ba7a58 data_size 0x2(0x85ea bytes) 2024-09-13T09:27:45.441 INFO:teuthology.orchestra.run.smithi063.stdout:5066 write 0x78b66d6 thru 0x78bf5b0 (0x8edb bytes) 2024-09-13T09:27:45.449 INFO:teuthology.orchestra.run.smithi063.stdout:5067 read 0x4591508 thru 0x459d93e (0xc437 bytes) 2024-09-13T09:27:45.465 INFO:teuthology.orchestra.run.smithi063.stdout:5068 write 0x8756a00 thru 0x875f667 (0x8c68 bytes) 2024-09-13T09:27:45.468 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.468 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.472 INFO:teuthology.orchestra.run.smithi063.stdout:5071 writesame 0x7a21bd9 thru 0x7a2da74 data_size 0x16(0xbe9c bytes) 2024-09-13T09:27:45.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:45.480 INFO:teuthology.orchestra.run.smithi063.stdout:5072 read 0x7df3796 thru 0x7df4744 (0xfaf bytes) 2024-09-13T09:27:45.501 INFO:teuthology.orchestra.run.smithi063.stdout:5073 write 0xd5e1ffb thru 0xd5ec9db (0xa9e1 bytes) 2024-09-13T09:27:45.506 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:45.503+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:45.510 INFO:teuthology.orchestra.run.smithi063.stdout:5077 clone 39 order 19 su 2048 sc 12 2024-09-13T09:27:45.586 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.586 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:45.607 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:45.604+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:45.708 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:45.700+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:45.749 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.749 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:45.808 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:45.804+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:45.862 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.862 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:45.910 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:45.907+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:45.979 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:45.980 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:45.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:46.010 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:46.004+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:46.111 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:46.107+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.143 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:46.212 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:46.208+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:46.259 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.259 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.269 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:46.316 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:46.312+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:46.385 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.386 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.396 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:46.397 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:27:46.444 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:46.440+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:46.510 INFO:teuthology.orchestra.run.smithi063.stdout:flattening image image_client.0-clone38 2024-09-13T09:27:46.513 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.513 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.515 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:46.511+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:46.525 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:46.616 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:46.611+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:46.631 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.631 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:46.693 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T09:27:46.717 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:46.715+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:46.752 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.752 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.763 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:46.772 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:46.819 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:46.812+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:46.873 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:46.874 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:46.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:46.913 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:46.919 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:46.912+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:46.933 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:46.951 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:46.972 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:46.988 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:46.988 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.423060242000005, 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:46.989 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:46.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:46.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:94707 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.823836149000002, 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018059941999999999, 2024-09-13T09:27:46.994 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 94707 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161317+0000", 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161315+0000", 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161324+0000", 2024-09-13T09:27:46.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4399999999999994e-06 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161344+0000", 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9786e-05 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161353+0000", 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3090000000000001e-06 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161406+0000", 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2120999999999998e-05 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.161468+0000", 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1921000000000006e-05 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.164261+0000", 2024-09-13T09:27:46.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027937190000000001 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179004+0000", 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014742496000000001 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179116+0000", 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011225799999999999 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179251+0000", 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000135158 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.179377+0000", 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012573700000000001 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.715490959999997, 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:47.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:47.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:47.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95867 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.177146006000001, 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.013993988000000001, 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95867 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.008 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808007+0000", 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808005+0000", 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808047+0000", 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2342999999999998e-05 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808085+0000", 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7755e-05 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808104+0000", 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9468000000000001e-05 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808178+0000", 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3516999999999994e-05 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808272+0000", 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4636000000000007e-05 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808477+0000", 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000204422 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820046+0000", 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011569364 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821939+0000", 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018930590000000001 2024-09-13T09:27:47.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.821977+0000", 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7790000000000002e-05 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822001+0000", 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4199999999999999e-05 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95869 3.12 3:4e5cc1e7:::rbd_data.1118298c62d6.0000000000000004:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.176987251, 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016770747999999999, 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95869 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.014 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808166+0000", 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808165+0000", 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808174+0000", 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.02e-06 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808192+0000", 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8294000000000001e-05 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808207+0000", 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4809e-05 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808264+0000", 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6974e-05 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808342+0000", 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8658000000000005e-05 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808504+0000", 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000161877 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822653+0000", 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014149198999999999 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824841+0000", 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021875900000000001 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824894+0000", 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2865000000000003e-05 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824937+0000", 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2658999999999999e-05 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95870 3.1f 3:faeb8947:::rbd_data.1118298c62d6.0000000000000005:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.176911310999998, 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.016686789, 2024-09-13T09:27:47.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95870 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808242+0000", 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808241+0000", 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808247+0000", 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5719999999999999e-06 2024-09-13T09:27:47.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808263+0000", 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5271e-05 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808275+0000", 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2418e-05 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808459+0000", 2024-09-13T09:27:47.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018368900000000001 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.808644+0000", 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000185689 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809084+0000", 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000439169 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820130+0000", 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011046791 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824864+0000", 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047339260000000003 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824896+0000", 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1868e-05 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.824929+0000", 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2364999999999999e-05 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:95868 3.10 e67/60 3:0adb1a16:::rbd_data.1118298c62d6.0000000000000003:head v 67'356, mlcod=65'353)", 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.175931867999999, 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0098441129999999998, 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.026 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809221+0000", 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809220+0000", 2024-09-13T09:27:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809230+0000", 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0018e-05 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809245+0000", 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4542000000000001e-05 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809254+0000", 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8069999999999994e-06 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809348+0000", 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3653000000000006e-05 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.809378+0000", 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0179e-05 2024-09-13T09:27:47.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819024+0000", 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0096453500000000005 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819065+0000", 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1668999999999997e-05 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.030 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95876 3.12 3:4dfe5062:::rbd_data.1118298c62d6.000000000000000b:head [stat,delete] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.168353148000001, 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010860465999999999, 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95876 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816800+0000", 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816798+0000", 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816807+0000", 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7669999999999992e-06 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816820+0000", 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2592000000000001e-05 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.816828+0000", 2024-09-13T09:27:47.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1869999999999997e-06 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.819951+0000", 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031233900000000002 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820037+0000", 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5892999999999999e-05 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.820206+0000", 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016905699999999999 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.822689+0000", 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024826539999999999 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827604+0000", 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049157330000000003 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827635+0000", 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0644e-05 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:51.827660+0000", 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5358999999999999e-05 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.036 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:107195 3.b 3:d1189f1e:::rbd_data.1118fd5aef74.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 12320768~5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 50.901607184, 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010711996999999999, 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 107195 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083546+0000", 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083544+0000", 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083935+0000", 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00039049700000000001 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083949+0000", 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3995000000000001e-05 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083955+0000", 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3110000000000002e-06 2024-09-13T09:27:47.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.083996+0000", 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0975e-05 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084114+0000", 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117785 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:47.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.084326+0000", 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021159 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.086124+0000", 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001798203 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094037+0000", 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0079126709999999996 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094133+0000", 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6337999999999998e-05 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:56.094258+0000", 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012506799999999999 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:47.042 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.953536153000002, 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114301, 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031617+0000", 2024-09-13T09:27:47.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031615+0000", 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031627+0000", 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2072999999999999e-05 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031658+0000", 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1037000000000001e-05 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031679+0000", 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0557e-05 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040706+0000", 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090265759999999997 2024-09-13T09:27:47.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040717+0000", 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1357e-05 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040731+0000", 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4355000000000001e-05 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop_reply(client.4376.0:125236 3.12 e82/81 ondisk, result = 0)", 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.953389281, 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.009114278, 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031764+0000", 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031763+0000", 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031769+0000", 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4659999999999996e-06 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031799+0000", 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9164000000000001e-05 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.031812+0000", 2024-09-13T09:27:47.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3465999999999999e-05 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040746+0000", 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0089339659999999998 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040752+0000", 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7830000000000004e-06 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:10.040878+0000", 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012623300000000001 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.835401863000001, 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:27:47.055 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128959 3.1f 3:fd5f8e4a:::rbd_data.11181500dc3e.000000000000000b:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.149417274000001, 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018621092999999998, 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.056 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128959 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835736+0000", 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835734+0000", 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835742+0000", 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1089999999999998e-06 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835759+0000", 2024-09-13T09:27:47.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6591000000000001e-05 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835766+0000", 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5860000000000001e-06 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835845+0000", 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9524000000000006e-05 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.835977+0000", 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013168399999999999 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.839738+0000", 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003761398 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854287+0000", 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014548442999999999 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854295+0000", 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5690000000000005e-06 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854329+0000", 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4198e-05 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.854357+0000", 2024-09-13T09:27:47.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7491000000000002e-05 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129111 3.1 3:87b12a70:::rbd_object_map.11181500dc3e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.858572589000001, 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086272250000000005, 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.062 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129111 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126581+0000", 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126580+0000", 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126586+0000", 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.178e-06 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126599+0000", 2024-09-13T09:27:47.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3124e-05 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.126609+0000", 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0294e-05 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.130890+0000", 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042807009999999996 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131001+0000", 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110368 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.131416+0000", 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00041545800000000001 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.134852+0000", 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0034356009999999999 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135120+0000", 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026872400000000002 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135173+0000", 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3149000000000002e-05 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.135208+0000", 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4297e-05 2024-09-13T09:27:47.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129312 3.1 3:875c0977:::rbd_data.11181500dc3e.00000000000000ea:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.445640710999999, 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0086345329999999998, 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129312 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539512+0000", 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539511+0000", 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539520+0000", 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0629999999999995e-06 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539535+0000", 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5242000000000001e-05 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539546+0000", 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1185999999999999e-05 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539603+0000", 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7358000000000002e-05 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.539688+0000", 2024-09-13T09:27:47.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4495999999999999e-05 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.541358+0000", 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016698030000000001 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548023+0000", 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0066649250000000004 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548069+0000", 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6715000000000003e-05 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548116+0000", 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6226000000000001e-05 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.548147+0000", 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1301999999999999e-05 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:129371 3.1f 3:fe73dde2:::rbd_data.11181500dc3e.000000000000000f:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.343297818, 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014271700999999999, 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.074 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 129371 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641855+0000", 2024-09-13T09:27:47.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641854+0000", 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641862+0000", 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5780000000000001e-06 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641877+0000", 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5327000000000001e-05 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641886+0000", 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4129999999999995e-06 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.641935+0000", 2024-09-13T09:27:47.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8775e-05 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.642014+0000", 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8574999999999999e-05 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.644683+0000", 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026697359999999998 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655832+0000", 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.011148371000000001 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.655974+0000", 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014196400000000001 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656079+0000", 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010503800000000001 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:25.656127+0000", 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.812e-05 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130769 3.1 3:84b2050c:::rbd_data.1118de6ae5af.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.950048115000001, 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024401849, 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.080 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130769 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035103+0000", 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035114+0000", 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1613e-05 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035136+0000", 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1472e-05 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035145+0000", 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4860000000000003e-06 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035204+0000", 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9027000000000003e-05 2024-09-13T09:27:47.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035363+0000", 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015868600000000001 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.040402+0000", 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050390000000000001 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059321+0000", 2024-09-13T09:27:47.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018919062 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059340+0000", 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9001999999999999e-05 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059432+0000", 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1535999999999999e-05 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059507+0000", 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5220000000000001e-05 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130832 3.12 3:4988a482:::rbd_data.1118de6ae5af.000000000000005e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 17.766439538, 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090196749999999996, 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.086 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130832 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218714+0000", 2024-09-13T09:27:47.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218710+0000", 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218726+0000", 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6439999999999998e-05 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218794+0000", 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7726999999999997e-05 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218815+0000", 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1449000000000001e-05 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.218935+0000", 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011945199999999999 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219123+0000", 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000187966 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.219363+0000", 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024062399999999999 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.222351+0000", 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029879989999999999 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227642+0000", 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052903109999999998 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227693+0000", 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1786000000000002e-05 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.227733+0000", 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9761000000000001e-05 2024-09-13T09:27:47.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135322 3.1b 3:d9011f0c:::rbd_data.11183e1674b5.000000000000009f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110)", 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:42.328576+0000", 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.6565771150000002, 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0090168950000000005, 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135322 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328576+0000", 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328576+0000", 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328575+0000", 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328584+0000", 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0799999999999995e-06 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328606+0000", 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2555e-05 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328619+0000", 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.294e-05 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328719+0000", 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010016 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328800+0000", 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0822999999999995e-05 2024-09-13T09:27:47.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.328869+0000", 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8977e-05 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.329639+0000", 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00077026199999999996 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.337467+0000", 2024-09-13T09:27:47.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0078272250000000002 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.337545+0000", 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8376999999999999e-05 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:42.337593+0000", 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7888e-05 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:135856 3.1b 3:de340d4c:::rbd_data.11183e1674b5.0000000000000019:head [delete] snapc 28=[28] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.545620+0000", 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.439533017, 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094166849999999993, 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 135856 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.545620+0000", 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.545620+0000", 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.545618+0000", 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:47.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.545630+0000", 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.182e-05 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.545660+0000", 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0474999999999999e-05 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.545676+0000", 2024-09-13T09:27:47.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5325e-05 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.546716+0000", 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001040771 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.546848+0000", 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000131552 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:47.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.547096+0000", 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024831499999999998 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.549234+0000", 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021374050000000002 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.554930+0000", 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056967049999999998 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.554990+0000", 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9880000000000003e-05 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.555037+0000", 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6631999999999997e-05 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.103 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:47.104 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:47.105 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:47.106 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:47.107 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:47.107 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:47.107 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:47.107 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:47.107 INFO:teuthology.orchestra.run.smithi063.stdout:5080 write 0x23c647b thru 0x23ce564 (0x80ea bytes) 2024-09-13T09:27:47.108 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:47.099+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:47.112 INFO:teuthology.orchestra.run.smithi063.stdout:5081 write 0xe7518a5 thru 0xe75269a (0xdf6 bytes) 2024-09-13T09:27:47.122 INFO:teuthology.orchestra.run.smithi063.stdout:5084 write 0x3cc95c8 thru 0x3cd2b02 (0x953b bytes) 2024-09-13T09:27:47.190 INFO:teuthology.orchestra.run.smithi063.stdout:5085 write 0x28a185c thru 0x28a4502 (0x2ca7 bytes) 2024-09-13T09:27:47.204 INFO:teuthology.orchestra.run.smithi063.stdout:5086 write 0x5ad8a5a thru 0x5adae89 (0x2430 bytes) 2024-09-13T09:27:47.205 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:47.200+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:47.221 INFO:teuthology.orchestra.run.smithi063.stdout:5087 writesame 0xe74eeef thru 0xe75c0d4 data_size 0x43(0xd1e6 bytes) 2024-09-13T09:27:47.241 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:47.241 INFO:teuthology.orchestra.run.smithi063.stdout:5090 read 0x558c175 thru 0x559282f (0x66bb bytes) 2024-09-13T09:27:47.244 INFO:teuthology.orchestra.run.smithi063.stdout:5093 writesame 0xb75855b thru 0xb75aa1b data_size 0x1(0x24c1 bytes) 2024-09-13T09:27:47.245 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:138131 3.6 e113/60 3:6175e419:::rbd_object_map.11181cfec05f:head v 113'410, mlcod=113'408)", 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.244335+0000", 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.00047741199999999998, 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00050369100000000001, 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244335+0000", 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244335+0000", 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244334+0000", 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244344+0000", 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0475000000000001e-05 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244365+0000", 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0437000000000001e-05 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244374+0000", 2024-09-13T09:27:47.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0219999999999993e-06 2024-09-13T09:27:47.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244423+0000", 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9336000000000003e-05 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.244437+0000", 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4248e-05 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:47.250 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 1 2024-09-13T09:27:47.251 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:47.259 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:47.271 INFO:teuthology.orchestra.run.smithi063.stdout:5094 write 0xa3e5791 thru 0xa3e5808 (0x78 bytes) 2024-09-13T09:27:47.277 INFO:teuthology.orchestra.run.smithi063.stdout:5095 write 0xc16a38 thru 0xc1eaa4 (0x806d bytes) 2024-09-13T09:27:47.301 INFO:teuthology.orchestra.run.smithi063.stdout:5096 read 0x5439207 thru 0x5448243 (0xf03d bytes) 2024-09-13T09:27:47.303 INFO:teuthology.orchestra.run.smithi063.stdout:5097 writesame 0xd5f0009 thru 0xd5f5839 data_size 0x1(0x5831 bytes) 2024-09-13T09:27:47.306 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:47.303+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:47.350 INFO:teuthology.orchestra.run.smithi063.stdout:5098 writesame 0x2241b1e thru 0x2244323 data_size 0x2f(0x2806 bytes) 2024-09-13T09:27:47.355 INFO:teuthology.orchestra.run.smithi063.stdout:5099 write 0x49daa4 thru 0x4a7ded (0xa34a bytes) 2024-09-13T09:27:47.401 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:47.401 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:47.401 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:47.401 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:47.402 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:47.407 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:47.400+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:47.411 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:27:47.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:47.429 INFO:teuthology.orchestra.run.smithi063.stdout:5103 write 0x88cfb0 thru 0x898684 (0xb6d5 bytes) 2024-09-13T09:27:47.452 INFO:teuthology.orchestra.run.smithi063.stdout:5104 write 0x2b05a1c thru 0x2b07223 (0x1808 bytes) 2024-09-13T09:27:47.484 INFO:teuthology.orchestra.run.smithi063.stdout:5105 writesame 0x7e95d17 thru 0x7e97b03 data_size 0x1(0x1ded bytes) 2024-09-13T09:27:47.498 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:47.498 INFO:teuthology.orchestra.run.smithi063.stdout:5107 write 0xa591044 thru 0xa597c6d (0x6c2a bytes) 2024-09-13T09:27:47.508 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:47.503+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:47.517 INFO:teuthology.orchestra.run.smithi063.stdout:5108 write 0xe2e7d60 thru 0xe2e9b3f (0x1de0 bytes) 2024-09-13T09:27:47.529 INFO:teuthology.orchestra.run.smithi063.stdout:5109 punch from 0x8e2b429 to 0x8e2d9a5, (0x257c bytes) 2024-09-13T09:27:47.529 INFO:teuthology.orchestra.run.smithi063.stdout:5110 read 0x2fba50a thru 0x2fbe3a7 (0x3e9e bytes) 2024-09-13T09:27:47.535 INFO:teuthology.orchestra.run.smithi063.stdout:5111 read 0x882f659 thru 0x883c533 (0xcedb bytes) 2024-09-13T09:27:47.544 INFO:teuthology.orchestra.run.smithi063.stdout:5113 write 0x5d73c9e thru 0x5d796d3 (0x5a36 bytes) 2024-09-13T09:27:47.562 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:47.562 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:47.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:4603 3.4 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 97.604933767999995, 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03115501, 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.563 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 4603 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953829+0000", 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.953827+0000", 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963305+0000", 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0094781810000000005 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963344+0000", 2024-09-13T09:27:47.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8859999999999997e-05 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963356+0000", 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2318e-05 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963493+0000", 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013671700000000001 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.963728+0000", 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023497000000000001 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.964424+0000", 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00069608699999999996 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984202+0000", 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019778334000000002 2024-09-13T09:27:47.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984918+0000", 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00071616699999999998 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984952+0000", 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3976999999999997e-05 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984984+0000", 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2141999999999998e-05 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:38921 3.6 3.e4988ec6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.912400566000002, 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03076023, 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 38921 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646362+0000", 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646371+0000", 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0010000000000007e-06 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646377+0000", 2024-09-13T09:27:47.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2920000000000001e-06 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.646382+0000", 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5920000000000002e-06 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677116+0000", 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030733864 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:36.677123+0000", 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7970000000000003e-06 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46584 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.439627383000001, 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039644561000000002, 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46584 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119136+0000", 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119133+0000", 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119152+0000", 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8680000000000001e-05 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119189+0000", 2024-09-13T09:27:47.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7527000000000002e-05 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119206+0000", 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6816999999999998e-05 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119306+0000", 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100345 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.119445+0000", 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000138992 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.126462+0000", 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0070165729999999999 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158628+0000", 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032166258000000003 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158679+0000", 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1143999999999997e-05 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158725+0000", 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5541000000000003e-05 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:38.158780+0000", 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5287000000000001e-05 2024-09-13T09:27:47.578 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69747 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1026048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009274386999998, 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030754513000000001, 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69747 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549489+0000", 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549488+0000", 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549492+0000", 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549501+0000", 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3440000000000001e-06 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549507+0000", 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7460000000000003e-06 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580130+0000", 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030622907000000001 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580178+0000", 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7670000000000003e-05 2024-09-13T09:27:47.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580243+0000", 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5227000000000004e-05 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69748 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 840704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009248513000003, 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030843282, 2024-09-13T09:27:47.583 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69748 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549515+0000", 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549514+0000", 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549518+0000", 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7890000000000001e-06 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549525+0000", 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5320000000000001e-06 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549532+0000", 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0079999999999996e-06 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580258+0000", 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030726203000000001 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580297+0000", 2024-09-13T09:27:47.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8111999999999997e-05 2024-09-13T09:27:47.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580358+0000", 2024-09-13T09:27:47.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1267999999999997e-05 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69749 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 1041408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009223149, 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030992519, 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69749 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549540+0000", 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549539+0000", 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549543+0000", 2024-09-13T09:27:47.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.817e-06 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549576+0000", 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2950000000000001e-05 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549615+0000", 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8631999999999999e-05 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580380+0000", 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030765150000000002 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580425+0000", 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5466999999999997e-05 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580532+0000", 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010710299999999999 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.593 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69750 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 856064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009138127, 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031088889000000001, 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.594 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69750 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549625+0000", 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549624+0000", 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549630+0000", 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4450000000000004e-06 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549644+0000", 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4489e-05 2024-09-13T09:27:47.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549652+0000", 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1009999999999992e-06 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580559+0000", 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030906772999999998 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580640+0000", 2024-09-13T09:27:47.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1499e-05 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580714+0000", 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3312000000000006e-05 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69752 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 871424~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.598 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009104731999997, 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031247700999999999, 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69752 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549658+0000", 2024-09-13T09:27:47.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549661+0000", 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.269e-06 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549666+0000", 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8389999999999997e-06 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549670+0000", 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580750+0000", 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031080059 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580822+0000", 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2117000000000001e-05 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580906+0000", 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4227000000000005e-05 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69755 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009090794000002, 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031463888000000002, 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.603 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69755 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549672+0000", 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549674+0000", 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.137e-06 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549678+0000", 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5810000000000001e-06 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549702+0000", 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3909000000000001e-05 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.580930+0000", 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031228926000000001 2024-09-13T09:27:47.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581002+0000", 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1227000000000001e-05 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581136+0000", 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013439800000000001 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69757 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009053047999998, 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031597021000000003, 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69757 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549710+0000", 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549712+0000", 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549720+0000", 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0630000000000006e-06 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549725+0000", 2024-09-13T09:27:47.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5179999999999998e-06 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581162+0000", 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031437158999999999 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581220+0000", 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7747000000000001e-05 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581307+0000", 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7096999999999996e-05 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69760 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 917504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009035132000001, 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031720941000000002, 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69760 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549728+0000", 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549730+0000", 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549734+0000", 2024-09-13T09:27:47.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8770000000000003e-06 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549737+0000", 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5090000000000002e-06 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581327+0000", 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031589821999999997 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581368+0000", 2024-09-13T09:27:47.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1084999999999997e-05 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581449+0000", 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0853000000000007e-05 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69765 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 932864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:47.616 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.009021484000002, 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031951479999999997, 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69765 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549742+0000", 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549741+0000", 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549744+0000", 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5399999999999998e-06 2024-09-13T09:27:47.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549768+0000", 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4048000000000001e-05 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549777+0000", 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2029999999999998e-06 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581465+0000", 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031687917000000003 2024-09-13T09:27:47.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581579+0000", 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011417300000000001 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581693+0000", 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113927 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69770 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 948224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008954838999998, 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032140488000000002, 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69770 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549808+0000", 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549784+0000", 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999762 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549836+0000", 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1623999999999999e-05 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549846+0000", 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8630000000000003e-06 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549851+0000", 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:27:47.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581729+0000", 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.031877579000000003 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581836+0000", 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107493 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581949+0000", 2024-09-13T09:27:47.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011246100000000001 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69775 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 963584~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008906779, 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032375690999999998, 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.625 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69775 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549856+0000", 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549858+0000", 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7020000000000001e-06 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549885+0000", 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6305999999999999e-05 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.627 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549908+0000", 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3458000000000001e-05 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.581979+0000", 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032070640999999997 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582090+0000", 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011182400000000001 2024-09-13T09:27:47.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582232+0000", 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014145499999999999 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69787 3.0 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:17 [read 978944~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008849507000001, 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032557330000000002, 2024-09-13T09:27:47.629 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69787 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549914+0000", 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549913+0000", 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549916+0000", 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0340000000000001e-06 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549922+0000", 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549928+0000", 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582269+0000", 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032340433000000002 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582340+0000", 2024-09-13T09:27:47.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1052999999999999e-05 2024-09-13T09:27:47.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582471+0000", 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013124100000000001 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69789 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008829398000003, 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032574741999999997, 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:47.634 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69789 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549934+0000", 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549933+0000", 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549937+0000", 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.67e-06 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549944+0000", 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2679999999999997e-06 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549950+0000", 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582501+0000", 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551068000000002 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582508+0000", 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3560000000000004e-06 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69792 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008807251999997, 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.03258001, 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69792 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549956+0000", 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549955+0000", 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549959+0000", 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7809999999999999e-06 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.549977+0000", 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7921999999999999e-05 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550000+0000", 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3557000000000002e-05 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582530+0000", 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032529258999999998 2024-09-13T09:27:47.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582536+0000", 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1210000000000003e-06 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69795 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008750984999999, 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032551878999999999, 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.642 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69795 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550012+0000", 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550011+0000", 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550015+0000", 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550021+0000", 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550025+0000", 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2719999999999999e-06 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582558+0000", 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032532806999999997 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582564+0000", 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:69797 3.0 3.c0749fe0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.008735088999998, 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032561458000000001, 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:47.646 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 69797 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550028+0000", 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550030+0000", 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0509999999999999e-06 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550034+0000", 2024-09-13T09:27:47.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.550037+0000", 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4450000000000001e-06 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582583+0000", 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.032546266999999997 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:45.582589+0000", 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9100000000000002e-06 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:136073 3.0 3:046f580d:::rbd_data.1118d1656b7c.0000000000000343:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 3.7473315459999998, 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.038067780000000002, 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:47.650 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 136073 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811431+0000", 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811437+0000", 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1569999999999997e-06 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811448+0000", 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1297999999999999e-05 2024-09-13T09:27:47.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811453+0000", 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4539999999999998e-06 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811531+0000", 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7268000000000006e-05 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.811730+0000", 2024-09-13T09:27:47.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019893499999999999 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 2", 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.819515+0000", 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0077851379999999996 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849352+0000", 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029836813 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:47.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849368+0000", 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6218e-05 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849423+0000", 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5417000000000002e-05 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.849499+0000", 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6029e-05 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.655 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.656 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:47.656 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:47.656 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:47.657 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:47.658 INFO:teuthology.orchestra.run.smithi063.stdout:5114 read 0xa1b659 thru 0xa260c2 (0xaa6a bytes) 2024-09-13T09:27:47.658 INFO:teuthology.orchestra.run.smithi063.stdout:5116 read 0x721fbf0 thru 0x7227f6c (0x837d bytes) 2024-09-13T09:27:47.658 INFO:teuthology.orchestra.run.smithi063.stdout:5117 punch from 0x1452c5 to 0x1529fa, (0xd735 bytes) 2024-09-13T09:27:47.658 INFO:teuthology.orchestra.run.smithi063.stdout:5118 punch from 0x8f15895 to 0x8f20e00, (0xb56b bytes) 2024-09-13T09:27:47.658 INFO:teuthology.orchestra.run.smithi063.stdout:5119 clone 40 order 18 su 1024 sc 15 2024-09-13T09:27:47.659 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:47.655+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:47.758 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:47.755+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:47.809 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:47.809 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:47.809 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:47.809 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:47.818 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:47.859 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:47.855+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:47.927 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:47.927 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:47.927 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:47.937 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:47.960 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:47.956+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:48.050 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:48.050 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:48.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:48.050 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:48.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.080031836000003, 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:48.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.006492823000002, 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:48.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.885888879000007, 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:48.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.871631096000002, 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:48.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:48.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.413987950000006, 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:48.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:48.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.208961901999999, 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:48.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:48.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 62.035222679, 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:48.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.841140236000001, 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:48.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.456777395000003, 2024-09-13T09:27:48.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:48.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:128958 3.13 e95/94 3:ca28514f:::rbd_data.11181500dc3e.000000000000000a:head v 95'531, mlcod=95'529)", 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.198194997000002, 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042980650000000002, 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847968+0000", 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850296+0000", 2024-09-13T09:27:48.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002328044 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850322+0000", 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5741e-05 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850334+0000", 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1936999999999999e-05 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850436+0000", 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000102631 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850488+0000", 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2169000000000001e-05 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852145+0000", 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016569480000000001 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852268+0000", 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012255999999999999 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128966 3.9 3:9420bb05:::rbd_data.11181500dc3e.000000000000008f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 23.157414066000001, 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053551420000000002, 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.099 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128966 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888748+0000", 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888764+0000", 2024-09-13T09:27:48.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6772000000000001e-05 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888807+0000", 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2827000000000002e-05 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888829+0000", 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1426000000000001e-05 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888965+0000", 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013610599999999999 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889174+0000", 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020964700000000001 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889456+0000", 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000281403 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.890649+0000", 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011928819999999999 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893917+0000", 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003268328 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894020+0000", 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103003 2024-09-13T09:27:48.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894106+0000", 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5986e-05 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130683 3.9 3:9160b546:::rbd_data.1118de6ae5af.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.235443872000001, 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005777351, 2024-09-13T09:27:48.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130683 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810718+0000", 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811313+0000", 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059520999999999999 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811352+0000", 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9453999999999999e-05 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811369+0000", 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6886999999999999e-05 2024-09-13T09:27:48.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811475+0000", 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105845 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811814+0000", 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033890599999999998 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812483+0000", 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00066937099999999999 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816386+0000", 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039026949999999999 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816411+0000", 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4916e-05 2024-09-13T09:27:48.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816465+0000", 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3835000000000002e-05 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816498+0000", 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3555e-05 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:133033 3.13 e101/94 3:c9a00536:::rbd_data.11182b78662d.000000000000001a:head v 101'625, mlcod=101'623)", 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.891460649000001, 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042112800000000004, 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154704+0000", 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.154702+0000", 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156468+0000", 2024-09-13T09:27:48.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017665440000000001 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156503+0000", 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4143000000000002e-05 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156518+0000", 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5543e-05 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156666+0000", 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014814599999999999 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.156733+0000", 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7037000000000002e-05 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.158851+0000", 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021177840000000002 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:33.158915+0000", 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4343000000000004e-05 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136071 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'381, mlcod=111'376)", 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.2358664639999999, 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048782419999999996, 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810306+0000", 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3920000000000003e-06 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810322+0000", 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810331+0000", 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4160000000000004e-06 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810388+0000", 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7163999999999997e-05 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810414+0000", 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6163999999999999e-05 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815117+0000", 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047025620000000004 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815177+0000", 2024-09-13T09:27:48.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9892999999999997e-05 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136080 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'385, mlcod=111'382)", 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.1836321490000001, 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057524539999999997, 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862531+0000", 2024-09-13T09:27:48.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:48.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862543+0000", 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1538e-05 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862560+0000", 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7156e-05 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862572+0000", 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1606e-05 2024-09-13T09:27:48.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862640+0000", 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8695999999999994e-05 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862669+0000", 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8354999999999999e-05 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868211+0000", 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055418450000000001 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868285+0000", 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4486000000000005e-05 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136101 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'391, mlcod=111'388)", 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:48.127 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.1403917330000004, 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050269909999999998, 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905772+0000", 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905783+0000", 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0822000000000001e-05 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905796+0000", 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3328e-05 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905805+0000", 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6079999999999996e-06 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905898+0000", 2024-09-13T09:27:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2958e-05 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905922+0000", 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4572000000000001e-05 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910718+0000", 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047962489999999998 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910800+0000", 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1749999999999995e-05 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136103 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'393, mlcod=111'388)", 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.1380229220000002, 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052766970000000003, 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.132 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908141+0000", 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908153+0000", 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2686e-05 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908170+0000", 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6651999999999999e-05 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908187+0000", 2024-09-13T09:27:48.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:27:48.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908236+0000", 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8959000000000003e-05 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908275+0000", 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8983999999999998e-05 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913370+0000", 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050951190000000004 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913419+0000", 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8293e-05 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.137 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136328 3.9 e111/107 3:934af333:::rbd_data.1118d1656b7c.000000000000000b:head v 111'255, mlcod=110'253)", 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.0371287850000002, 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044988249999999997, 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009035+0000", 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009276+0000", 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024107299999999999 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009292+0000", 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6396e-05 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009300+0000", 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6559999999999992e-06 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009375+0000", 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4899000000000002e-05 2024-09-13T09:27:48.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009402+0000", 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7404999999999999e-05 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.013491+0000", 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040891260000000002 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.013535+0000", 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3401e-05 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:138187 3.0 e113/21 3:05ffe32b:::rbd_data.11181cfec05f.000000000000001c:head v 113'771, mlcod=113'769)", 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.75329100800000004, 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051940129999999999, 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.142 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292873+0000", 2024-09-13T09:27:48.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292894+0000", 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1759999999999998e-05 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292910+0000", 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5707000000000001e-05 2024-09-13T09:27:48.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292917+0000", 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9909999999999996e-06 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292970+0000", 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2494999999999998e-05 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292996+0000", 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6027e-05 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.298020+0000", 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050239480000000003 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.298068+0000", 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8189999999999998e-05 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:141269 3.0 e114/21 3:03e78e0b:::rbd_data.11181cfec05f.00000000000000ed:head v 114'774, mlcod=114'772)", 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.43364915900000001, 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053503429999999996, 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:48.147 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612515+0000", 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612522+0000", 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4320000000000004e-06 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612530+0000", 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7579999999999998e-06 2024-09-13T09:27:48.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612536+0000", 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9560000000000002e-06 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612626+0000", 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9307000000000004e-05 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:48.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612646+0000", 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0349000000000001e-05 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.617820+0000", 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005173731 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.617866+0000", 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6325000000000001e-05 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.152 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.152 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:48.152 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:48.152 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:48.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:48.154 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:48.152+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:48.254 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:48.251+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:48.271 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:48.271 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:48.271 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:48.271 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:48.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:48.355 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:48.351+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:48.389 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:48.389 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:48.389 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:48.389 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:48.389 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:48.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:48.456 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:48.452+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:48.543 INFO:teuthology.orchestra.run.smithi063.stdout:flattening image image_client.0-clone39 2024-09-13T09:27:48.557 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:48.555+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.863008413000003, 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:48.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.565 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862985899000002, 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:48.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862970062000002, 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:48.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:48.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862773118, 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:48.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:48.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862748793999998, 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:48.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:48.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:48.583 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862732334, 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:48.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862716313, 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:48.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:48.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:48.592 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862665980000003, 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:48.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.862562404999998, 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:48.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853184609000003, 2024-09-13T09:27:48.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:48.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:48.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.605 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853165945000001, 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:48.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:48.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853149455000001, 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:48.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.614 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853133669999998, 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:48.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853117548999997, 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:48.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853101643999999, 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:48.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853085174999997, 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:48.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853063323000001, 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853046507999998, 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:48.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:48.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853030210999997, 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.853014610000002, 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:48.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:48.650 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:48.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:48.697 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:48.695+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:48.758 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:48.755+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:48.767 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:48.767 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:48.767 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:48.767 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:48.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:48.859 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:48.852+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:48.875 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:48.876 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:48.876 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:48.876 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:48.876 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:48.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:48.960 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:48.955+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:48.996 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634246906999998, 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634231448000001, 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634215368, 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634199594999998, 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634177694000002, 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634161808000002, 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634146493000003, 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634130448000001, 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634114724, 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634100124, 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634084696000002, 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.634068810000002, 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.284941566999997, 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.284807200000003, 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.284786308999998, 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.910736704000001, 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.909859144000002, 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.909819583999997, 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.687496113000002, 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:27:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:27:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.160282907000003, 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:27:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:27:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:49.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:49.091 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:49.092 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-13T09:27:49.093 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:49.093 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:49.091+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:49.108 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:49.193 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:49.188+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:49.248 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:49.248 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:49.248 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:49.248 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:49.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:49.259 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:49.285 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:49.295 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:49.288+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:49.305 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:49.323 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:49.345 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:49.359 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:49.359 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:49.359 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:49.359 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:49.359 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:49.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:49.369 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:49.387 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:49.396 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:49.391+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:49.408 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:49.411 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:49.412 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:49.420 INFO:teuthology.orchestra.run.smithi063.stdout:5120 trunc from 0xe75c0d5 to 0xb14202 2024-09-13T09:27:49.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:27:49.497 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:49.491+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:49.507 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:49.508 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:49.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:49.597 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:49.595+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:49.657 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:49.657 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:49.667 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:49.698 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:49.692+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:49.791 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:49.799 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:49.792+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:49.801 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:49.900 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:49.894+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:49.909 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:49.909 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:49.918 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:49.927 INFO:teuthology.orchestra.run.smithi063.stdout:5121 write 0x99d4ed5 thru 0x99d95d3 (0x46ff bytes) 2024-09-13T09:27:49.959 INFO:teuthology.orchestra.run.smithi063.stdout:5122 writesame 0x8164cea thru 0x8174639 data_size 0x8(0xf950 bytes) 2024-09-13T09:27:50.001 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:49.996+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:50.001 INFO:teuthology.orchestra.run.smithi063.stdout:5123 write 0xceccdd1 thru 0xced6715 (0x9945 bytes) 2024-09-13T09:27:50.049 INFO:teuthology.orchestra.run.smithi063.stdout:5124 trunc from 0xced6716 to 0x258775b 2024-09-13T09:27:50.064 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.064 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.074 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:50.084 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:27:50.084 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:27:50.088 INFO:teuthology.orchestra.run.smithi063.stdout:5126 write 0x343db3c thru 0x3444ea5 (0x736a bytes) 2024-09-13T09:27:50.102 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:50.098+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:50.116 INFO:teuthology.orchestra.run.smithi063.stdout:5127 read 0xc93077 thru 0xc9f96a (0xc8f4 bytes) 2024-09-13T09:27:50.121 INFO:teuthology.orchestra.run.smithi063.stdout:5128 trunc from 0x3444ea6 to 0xbb1feb 2024-09-13T09:27:50.149 INFO:teuthology.orchestra.run.smithi063.stdout:5129 trunc from 0xbb1feb to 0x4606e1b 2024-09-13T09:27:50.184 INFO:teuthology.orchestra.run.smithi063.stdout:5130 write 0xb54e63b thru 0xb55973f (0xb105 bytes) 2024-09-13T09:27:50.203 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:50.196+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:50.208 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.208 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.209 INFO:teuthology.orchestra.run.smithi063.stdout:5131 read 0x36ca1a0 thru 0x36d6a0a (0xc86b bytes) 2024-09-13T09:27:50.209 INFO:teuthology.orchestra.run.smithi063.stdout:5132 read 0x711b8ba thru 0x711ece1 (0x3428 bytes) 2024-09-13T09:27:50.209 INFO:teuthology.orchestra.run.smithi063.stdout:5133 read 0xb47718c thru 0xb478177 (0xfec bytes) 2024-09-13T09:27:50.210 INFO:teuthology.orchestra.run.smithi063.stdout:5135 read 0x2182354 thru 0x2186ef7 (0x4ba4 bytes) 2024-09-13T09:27:50.210 INFO:teuthology.orchestra.run.smithi063.stdout:5136 write 0x27a081e thru 0x27ae94e (0xe131 bytes) 2024-09-13T09:27:50.222 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:50.237 INFO:teuthology.orchestra.run.smithi063.stdout:5139 trunc from 0xb559740 to 0x7b26db2 2024-09-13T09:27:50.279 INFO:teuthology.orchestra.run.smithi063.stdout:5140 writesame 0xe4d8421 thru 0xe4dc04a data_size 0x22(0x3c2a bytes) 2024-09-13T09:27:50.279 INFO:teuthology.orchestra.run.smithi063.stdout:5141 trunc from 0xe4dc04b to 0xe98a801 2024-09-13T09:27:50.304 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:50.298+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:50.313 INFO:teuthology.orchestra.run.smithi063.stdout:5143 trunc from 0xe98a801 to 0x945e97 2024-09-13T09:27:50.345 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.345 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.357 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:50.361 INFO:teuthology.orchestra.run.smithi063.stdout:5144 write 0x88d07cc thru 0x88dd5a1 (0xcdd6 bytes) 2024-09-13T09:27:50.387 INFO:teuthology.orchestra.run.smithi063.stdout:5145 write 0x80c5c60 thru 0x80c8a64 (0x2e05 bytes) 2024-09-13T09:27:50.403 INFO:teuthology.orchestra.run.smithi063.stdout:5146 writesame 0xdd2d1f thru 0xdd342b data_size 0x5(0x70d bytes) 2024-09-13T09:27:50.404 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:50.400+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:50.410 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.411 INFO:teuthology.orchestra.run.smithi063.stdout:5149 trunc from 0x88dd5a2 to 0x787e9e6 2024-09-13T09:27:50.439 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:27:50.447 INFO:teuthology.orchestra.run.smithi063.stdout:5150 writesame 0x2a65b91 thru 0x2a72eaf data_size 0x7(0xd31f bytes) 2024-09-13T09:27:50.475 INFO:teuthology.orchestra.run.smithi063.stdout:5153 write 0x3a1a33e thru 0x3a1ca32 (0x26f5 bytes) 2024-09-13T09:27:50.494 INFO:teuthology.orchestra.run.smithi063.stdout:5154 punch from 0x5e43dc1 to 0x5e4d150, (0x938f bytes) 2024-09-13T09:27:50.495 INFO:teuthology.orchestra.run.smithi063.stdout:5155 punch from 0x4e7677f to 0x4e78b4c, (0x23cd bytes) 2024-09-13T09:27:50.495 INFO:teuthology.orchestra.run.smithi063.stdout:5156 writesame 0x76f3633 thru 0x76fac88 data_size 0x99(0x7656 bytes) 2024-09-13T09:27:50.504 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.504 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.504 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:50.498+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:50.515 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:50.516 INFO:teuthology.orchestra.run.smithi063.stdout:5157 write 0x2964c43 thru 0x297442b (0xf7e9 bytes) 2024-09-13T09:27:50.520 INFO:teuthology.orchestra.run.smithi063.stdout:5158 write 0x50f9206 thru 0x51064bf (0xd2ba bytes) 2024-09-13T09:27:50.550 INFO:teuthology.orchestra.run.smithi063.stdout:5159 trunc from 0x787e9e6 to 0x312a3f7 2024-09-13T09:27:50.594 INFO:teuthology.orchestra.run.smithi063.stdout:5161 trunc from 0x312a3f7 to 0xa364318 2024-09-13T09:27:50.604 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.604 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.605 INFO:teuthology.orchestra.run.smithi063.stdout:5164 writesame 0xa319803 thru 0xa31e638 data_size 0x6(0x4e36 bytes) 2024-09-13T09:27:50.605 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:50.598+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:50.625 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.626 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.633 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.640 INFO:teuthology.orchestra.run.smithi063.stdout:5167 trunc from 0xa364318 to 0x5d71ac4 2024-09-13T09:27:50.641 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:50.661 INFO:teuthology.orchestra.run.smithi063.stdout:5169 write 0x1617982 thru 0x1626cd2 (0xf351 bytes) 2024-09-13T09:27:50.692 INFO:teuthology.orchestra.run.smithi063.stdout:5170 writesame 0x310fbc7 thru 0x3116a94 data_size 0xd(0x6ece bytes) 2024-09-13T09:27:50.706 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:50.700+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:50.734 INFO:teuthology.orchestra.run.smithi063.stdout:5171 trunc from 0x5d71ac4 to 0xe5df02b 2024-09-13T09:27:50.747 INFO:teuthology.orchestra.run.smithi063.stdout:5172 writesame 0xbf49351 thru 0xbf52fba data_size 0x1(0x9c6a bytes) 2024-09-13T09:27:50.776 INFO:teuthology.orchestra.run.smithi063.stdout:5174 writesame 0x8d3c0c3 thru 0x8d47dfd data_size 0x1(0xbd3b bytes) 2024-09-13T09:27:50.784 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.784 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.797 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:50.802 INFO:teuthology.orchestra.run.smithi063.stdout:5175 read 0x8976229 thru 0x89860e3 (0xfebb bytes) 2024-09-13T09:27:50.807 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:50.802+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:50.822 INFO:teuthology.orchestra.run.smithi063.stdout:5176 trunc from 0xe5df02b to 0xe05c11 2024-09-13T09:27:50.870 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.885 INFO:teuthology.orchestra.run.smithi063.stdout:5178 trunc from 0xe05c11 to 0xab8e7d4 2024-09-13T09:27:50.898 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.907 INFO:teuthology.orchestra.run.smithi063.stdout:5181 trunc from 0xab8e7d4 to 0x509b2e3 2024-09-13T09:27:50.907 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:50.900+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:50.921 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:50.921 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:50.932 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:50.933 INFO:teuthology.orchestra.run.smithi063.stdout:5182 write 0x51a9a60 thru 0x51b684a (0xcdeb bytes) 2024-09-13T09:27:50.961 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:50.982 INFO:teuthology.orchestra.run.smithi063.stdout:5184 write 0xb1eb4fd thru 0xb1edaa3 (0x25a7 bytes) 2024-09-13T09:27:51.007 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.008 INFO:teuthology.orchestra.run.smithi063.stdout:5186 trunc from 0xb1edaa4 to 0xb8d29ac 2024-09-13T09:27:51.008 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:51.002+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:51.018 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.018 INFO:teuthology.orchestra.run.smithi063.stdout:5189 read 0x52f176e thru 0x52f5171 (0x3a04 bytes) 2024-09-13T09:27:51.034 INFO:teuthology.orchestra.run.smithi063.stdout:5190 writesame 0xd215804 thru 0xd2206b2 data_size 0x1(0xaeaf bytes) 2024-09-13T09:27:51.035 INFO:teuthology.orchestra.run.smithi063.stdout:5191 punch from 0x1ab0b93 to 0x1ab6e04, (0x6271 bytes) 2024-09-13T09:27:51.035 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.035 INFO:teuthology.orchestra.run.smithi063.stdout:5193 read 0xce1dbee thru 0xce21cda (0x40ed bytes) 2024-09-13T09:27:51.035 INFO:teuthology.orchestra.run.smithi063.stdout:5195 read 0x15b3bd1 thru 0x15c2022 (0xe452 bytes) 2024-09-13T09:27:51.035 INFO:teuthology.orchestra.run.smithi063.stdout:5196 read 0xb59a813 thru 0xb59b434 (0xc22 bytes) 2024-09-13T09:27:51.035 INFO:teuthology.orchestra.run.smithi063.stdout:5199 writesame 0x4c2a750 thru 0x4c3857b data_size 0x4(0xde2c bytes) 2024-09-13T09:27:51.058 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:51.058 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:51.068 INFO:teuthology.orchestra.run.smithi063.stdout:5200 trunc from 0xd2206b3 to 0xe3aaabf 2024-09-13T09:27:51.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:51.078 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.078 INFO:teuthology.orchestra.run.smithi063.stdout:5204 writesame 0x856d4aa thru 0x8577a27 data_size 0x45(0xa57e bytes) 2024-09-13T09:27:51.104 INFO:teuthology.orchestra.run.smithi063.stdout:5205 write 0x8411949 thru 0x841beae (0xa566 bytes) 2024-09-13T09:27:51.109 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:51.104+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:51.111 INFO:teuthology.orchestra.run.smithi063.stdout:5206 punch from 0x2b39831 to 0x2b45fc0, (0xc78f bytes) 2024-09-13T09:27:51.111 INFO:teuthology.orchestra.run.smithi063.stdout:5208 punch from 0xb81396c to 0xb81a17e, (0x6812 bytes) 2024-09-13T09:27:51.111 INFO:teuthology.orchestra.run.smithi063.stdout:5211 punch from 0x4f22af6 to 0x4f3055e, (0xda68 bytes) 2024-09-13T09:27:51.111 INFO:teuthology.orchestra.run.smithi063.stdout:5213 writesame 0x46cc35b thru 0x46d8088 data_size 0x3a(0xbd2e bytes) 2024-09-13T09:27:51.138 INFO:teuthology.orchestra.run.smithi063.stdout:5214 trunc from 0xe3aaabf to 0xb5e6792 2024-09-13T09:27:51.148 INFO:teuthology.orchestra.run.smithi063.stdout:5215 read 0x8962e11 thru 0x896ff8d (0xd17d bytes) 2024-09-13T09:27:51.148 INFO:teuthology.orchestra.run.smithi063.stdout:5216 write 0x1cc9896 thru 0x1cd5a4d (0xc1b8 bytes) 2024-09-13T09:27:51.182 INFO:teuthology.orchestra.run.smithi063.stdout:5217 read 0x1a1e77f thru 0x1a29521 (0xada3 bytes) 2024-09-13T09:27:51.182 INFO:teuthology.orchestra.run.smithi063.stdout:5218 punch from 0x57fdd2 to 0x5846fc, (0x492a bytes) 2024-09-13T09:27:51.193 INFO:teuthology.orchestra.run.smithi063.stdout:5220 trunc from 0xb5e6792 to 0x53e1a60 2024-09-13T09:27:51.195 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:51.195 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:51.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:51.219 INFO:teuthology.orchestra.run.smithi063.stdout:5223 punch from 0x9fd8e5 to 0x9febea, (0x1305 bytes) 2024-09-13T09:27:51.219 INFO:teuthology.orchestra.run.smithi063.stdout:5225 punch from 0x2663ffc to 0x2664bca, (0xbce bytes) 2024-09-13T09:27:51.219 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:51.212+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:51.234 INFO:teuthology.orchestra.run.smithi063.stdout:5226 write 0x855203b thru 0x8553519 (0x14df bytes) 2024-09-13T09:27:51.249 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.249 INFO:teuthology.orchestra.run.smithi063.stdout:5229 read 0x714cba thru 0x72116a (0xc4b1 bytes) 2024-09-13T09:27:51.310 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:51.304+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:51.336 INFO:teuthology.orchestra.run.smithi063.stdout:5230 punch from 0x9c2c58 to 0x9cd879, (0xac21 bytes) 2024-09-13T09:27:51.365 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:51.365 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:51.377 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:51.412 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:51.406+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:51.412 INFO:teuthology.orchestra.run.smithi063.stdout:5231 write 0x891a96d thru 0x8925dbc (0xb450 bytes) 2024-09-13T09:27:51.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:51.449 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.449 INFO:teuthology.orchestra.run.smithi063.stdout:5235 read 0x25a365f thru 0x25ad393 (0x9d35 bytes) 2024-09-13T09:27:51.450 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:27:51.461 INFO:teuthology.orchestra.run.smithi063.stdout:5236 writesame 0xaa6074d thru 0xaa68736 data_size 0x7(0x7fea bytes) 2024-09-13T09:27:51.461 INFO:teuthology.orchestra.run.smithi063.stdout:5237 punch from 0xa60e6ea to 0xa61b7da, (0xd0f0 bytes) 2024-09-13T09:27:51.464 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:27:51.479 INFO:teuthology.orchestra.run.smithi063.stdout:5238 write 0xe33a71d thru 0xe3423ba (0x7c9e bytes) 2024-09-13T09:27:51.506 INFO:teuthology.orchestra.run.smithi063.stdout:5240 read 0xd0f6daa thru 0xd104b6d (0xddc4 bytes) 2024-09-13T09:27:51.506 INFO:teuthology.orchestra.run.smithi063.stdout:5241 write 0x6f19bc thru 0x6f7c8c (0x62d1 bytes) 2024-09-13T09:27:51.513 INFO:teuthology.orchestra.run.smithi063.stdout:5242 read 0x8251a78 thru 0x825d82f (0xbdb8 bytes) 2024-09-13T09:27:51.513 INFO:teuthology.orchestra.run.smithi063.stdout:5244 punch from 0xde1da10 to 0xde21975, (0x3f65 bytes) 2024-09-13T09:27:51.513 INFO:teuthology.orchestra.run.smithi063.stdout:5245 write 0x7290e2 thru 0x733f7d (0xae9c bytes) 2024-09-13T09:27:51.513 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:51.506+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:51.520 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.529 INFO:teuthology.orchestra.run.smithi063.stdout:5247 writesame 0xe909d93 thru 0xe912126 data_size 0x2a(0x8394 bytes) 2024-09-13T09:27:51.529 INFO:teuthology.orchestra.run.smithi063.stdout:5248 read 0x5fbbe20 thru 0x5fca1c7 (0xe3a8 bytes) 2024-09-13T09:27:51.540 INFO:teuthology.orchestra.run.smithi063.stdout:5249 trunc from 0xe912127 to 0x8b53127 2024-09-13T09:27:51.564 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:51.564 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:51.570 INFO:teuthology.orchestra.run.smithi063.stdout:5250 read 0x76403a9 thru 0x764db7f (0xd7d7 bytes) 2024-09-13T09:27:51.570 INFO:teuthology.orchestra.run.smithi063.stdout:5252 punch from 0x74fc20c to 0x7502781, (0x6575 bytes) 2024-09-13T09:27:51.576 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:51.581 INFO:teuthology.orchestra.run.smithi063.stdout:5253 writesame 0x8f61532 thru 0x8f6ebcf data_size 0x3(0xd69e bytes) 2024-09-13T09:27:51.581 INFO:teuthology.orchestra.run.smithi063.stdout:5254 read 0x1105fae thru 0x110b23e (0x5291 bytes) 2024-09-13T09:27:51.582 INFO:teuthology.orchestra.run.smithi063.stdout:5255 punch from 0x7268460 to 0x726bf2c, (0x3acc bytes) 2024-09-13T09:27:51.582 INFO:teuthology.orchestra.run.smithi063.stdout:5257 writesame 0x7bf5f1e thru 0x7bf99d1 data_size 0x4(0x3ab4 bytes) 2024-09-13T09:27:51.614 INFO:teuthology.orchestra.run.smithi063.stdout:5258 read 0x41f4182 thru 0x41f46e3 (0x562 bytes) 2024-09-13T09:27:51.614 INFO:teuthology.orchestra.run.smithi063.stdout:5259 punch from 0x4253bd3 to 0x4262c99, (0xf0c6 bytes) 2024-09-13T09:27:51.615 INFO:teuthology.orchestra.run.smithi063.stdout:5260 punch from 0x8d761d8 to 0x8d7a4e2, (0x430a bytes) 2024-09-13T09:27:51.615 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.615 INFO:teuthology.orchestra.run.smithi063.stdout:5262 punch from 0x1b16354 to 0x1b220df, (0xbd8b bytes) 2024-09-13T09:27:51.615 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:51.608+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:51.621 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:51.627 INFO:teuthology.orchestra.run.smithi063.stdout:5263 trunc from 0x8f6ebd0 to 0x63041d 2024-09-13T09:27:51.651 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:51.679 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:51.701 INFO:teuthology.orchestra.run.smithi063.stdout:5264 writesame 0x28fa0a thru 0x29916b data_size 0x9(0x9762 bytes) 2024-09-13T09:27:51.716 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:51.717 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:51.714+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:51.727 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:51.727 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:51.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:51.727 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 101.753853107, 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:51.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:27:51.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.732 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4614 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'107, mlcod=35'105)", 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 101.680314094, 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042678899999999999, 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039672+0000", 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039671+0000", 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039689+0000", 2024-09-13T09:27:51.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7892000000000001e-05 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039710+0000", 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0665e-05 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039719+0000", 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2979999999999997e-06 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039858+0000", 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000139402 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.039901+0000", 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2404000000000003e-05 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043874+0000", 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039733349999999997 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:10.043940+0000", 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5959999999999999e-05 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26519 3.0 e46/21 3:0599eb57:::rbd_data.111812ca204d.000000000000000f:head v 46'103, mlcod=44'101)", 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.559710150000001, 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043095520000000003, 2024-09-13T09:27:51.737 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160276+0000", 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.160274+0000", 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161593+0000", 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013187559999999999 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161629+0000", 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6118999999999998e-05 2024-09-13T09:27:51.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161640+0000", 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.093e-05 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161731+0000", 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1689999999999998e-05 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.161777+0000", 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5827e-05 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164519+0000", 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027415299999999998 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.164586+0000", 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6760000000000005e-05 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.741 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:26524 3.9 e46/21 3:950db318:::rbd_data.111812ca204d.0000000000000012:head v 46'69, mlcod=44'67)", 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.545452366999996, 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042364960000000002, 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174534+0000", 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.174533+0000", 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176212+0000", 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016787760000000001 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176232+0000", 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0316999999999999e-05 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176241+0000", 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4600000000000003e-06 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176327+0000", 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6604e-05 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.176363+0000", 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5598e-05 2024-09-13T09:27:51.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178675+0000", 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023125039999999999 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:24.178770+0000", 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4838000000000002e-05 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.087809221000001, 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:27:51.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.751 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 73.882783173000007, 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:27:51.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.709043949999995, 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.756 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:27:51.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.760 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 65.514961506999995, 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:51.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:27:51.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:27:51.765 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.130598665999997, 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:27:51.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:128958 3.13 e95/94 3:ca28514f:::rbd_data.11181500dc3e.000000000000000a:head v 95'531, mlcod=95'529)", 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.872016267999999, 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042980650000000002, 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.770 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847970+0000", 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.847968+0000", 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850296+0000", 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002328044 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850322+0000", 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5741e-05 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850334+0000", 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1936999999999999e-05 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850436+0000", 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000102631 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.850488+0000", 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2169000000000001e-05 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852145+0000", 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016569480000000001 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.852268+0000", 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012255999999999999 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.774 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128966 3.9 3:9420bb05:::rbd_data.11181500dc3e.000000000000008f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.831235336999999, 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053551420000000002, 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:51.775 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128966 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888748+0000", 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888764+0000", 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6772000000000001e-05 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888807+0000", 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2827000000000002e-05 2024-09-13T09:27:51.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888829+0000", 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1426000000000001e-05 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888965+0000", 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013610599999999999 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889174+0000", 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020964700000000001 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889456+0000", 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000281403 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.890649+0000", 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011928819999999999 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893917+0000", 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003268328 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894020+0000", 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103003 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894106+0000", 2024-09-13T09:27:51.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5986e-05 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130683 3.9 3:9160b546:::rbd_data.1118de6ae5af.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 22.909265142999999, 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005777351, 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:51.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130683 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810718+0000", 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811313+0000", 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059520999999999999 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811352+0000", 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9453999999999999e-05 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811369+0000", 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6886999999999999e-05 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811475+0000", 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105845 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811814+0000", 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033890599999999998 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812483+0000", 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00066937099999999999 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816386+0000", 2024-09-13T09:27:51.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039026949999999999 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816411+0000", 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4916e-05 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816465+0000", 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3835000000000002e-05 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816498+0000", 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3555e-05 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136071 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'381, mlcod=111'376)", 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.9096877350000003, 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048782419999999996, 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.787 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810306+0000", 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3920000000000003e-06 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810322+0000", 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810331+0000", 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4160000000000004e-06 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810388+0000", 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7163999999999997e-05 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810414+0000", 2024-09-13T09:27:51.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6163999999999999e-05 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815117+0000", 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047025620000000004 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815177+0000", 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9892999999999997e-05 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136080 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'385, mlcod=111'382)", 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.8574534199999997, 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057524539999999997, 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862531+0000", 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862543+0000", 2024-09-13T09:27:51.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1538e-05 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862560+0000", 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7156e-05 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862572+0000", 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1606e-05 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862640+0000", 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8695999999999994e-05 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862669+0000", 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8354999999999999e-05 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868211+0000", 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055418450000000001 2024-09-13T09:27:51.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868285+0000", 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4486000000000005e-05 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136101 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'391, mlcod=111'388)", 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.814213004, 2024-09-13T09:27:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050269909999999998, 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905772+0000", 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905783+0000", 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0822000000000001e-05 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905796+0000", 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3328e-05 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905805+0000", 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6079999999999996e-06 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905898+0000", 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2958e-05 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905922+0000", 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4572000000000001e-05 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910718+0000", 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047962489999999998 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910800+0000", 2024-09-13T09:27:51.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1749999999999995e-05 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136103 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'393, mlcod=111'388)", 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.8118441929999998, 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052766970000000003, 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.801 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908141+0000", 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908153+0000", 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2686e-05 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908170+0000", 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6651999999999999e-05 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908187+0000", 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908236+0000", 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8959000000000003e-05 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908275+0000", 2024-09-13T09:27:51.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8983999999999998e-05 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913370+0000", 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050951190000000004 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913419+0000", 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8293e-05 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.805 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136328 3.9 e111/107 3:934af333:::rbd_data.1118d1656b7c.000000000000000b:head v 111'255, mlcod=110'253)", 2024-09-13T09:27:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:51.808 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 7.7109500559999997, 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044988249999999997, 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009036+0000", 2024-09-13T09:27:51.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009035+0000", 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009276+0000", 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024107299999999999 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009292+0000", 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6396e-05 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009300+0000", 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6559999999999992e-06 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009375+0000", 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4899000000000002e-05 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.009402+0000", 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7404999999999999e-05 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.013491+0000", 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0040891260000000002 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:44.013535+0000", 2024-09-13T09:27:51.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3401e-05 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:138187 3.0 e113/21 3:05ffe32b:::rbd_data.11181cfec05f.000000000000001c:head v 113'771, mlcod=113'769)", 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.4271122790000002, 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051940129999999999, 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292873+0000", 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:51.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292894+0000", 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1759999999999998e-05 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292910+0000", 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5707000000000001e-05 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292917+0000", 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9909999999999996e-06 2024-09-13T09:27:51.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292970+0000", 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2494999999999998e-05 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292996+0000", 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6027e-05 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.298020+0000", 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050239480000000003 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.298068+0000", 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8189999999999998e-05 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:141269 3.0 e114/21 3:03e78e0b:::rbd_data.11181cfec05f.00000000000000ed:head v 114'774, mlcod=114'772)", 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:27:51.817 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.1074704300000002, 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053503429999999996, 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612515+0000", 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612522+0000", 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4320000000000004e-06 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612530+0000", 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7579999999999998e-06 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612536+0000", 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9560000000000002e-06 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612626+0000", 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9307000000000004e-05 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612646+0000", 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0349000000000001e-05 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.617820+0000", 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005173731 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.617866+0000", 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6325000000000001e-05 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.821 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:143378 3.4 e115/21 3:232b5a0d:::rbd_data.11181cfec05f.00000000000000f2:head v 115'577, mlcod=115'574)", 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:48.736295+0000", 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 2.983691082, 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062177999999999999, 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736295+0000", 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736295+0000", 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736294+0000", 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736361+0000", 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6675999999999997e-05 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736381+0000", 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0720000000000002e-05 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736389+0000", 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.089e-06 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736450+0000", 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0544000000000002e-05 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736477+0000", 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7608000000000001e-05 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.742468+0000", 2024-09-13T09:27:51.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059905610000000001 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.742513+0000", 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4790999999999999e-05 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:51.826 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:51.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:51.828 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:51.828 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:51.829 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:51.820+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:51.851 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:51.863 INFO:teuthology.orchestra.run.smithi063.stdout:5265 trunc from 0x63041d to 0x91d0c29 2024-09-13T09:27:51.872 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.872 INFO:teuthology.orchestra.run.smithi063.stdout:5270 read 0x8b4c39 thru 0x8ba3e8 (0x57b0 bytes) 2024-09-13T09:27:51.873 INFO:teuthology.orchestra.run.smithi063.stdout:5273 writesame 0x1a13bf9 thru 0x1a23b86 data_size 0x7(0xff8e bytes) 2024-09-13T09:27:51.876 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:51.878 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:51.878 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:51.878 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:51.878 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:51.879 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:51.879 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:51.879 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:51.879 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:51.897 INFO:teuthology.orchestra.run.smithi063.stdout:5274 writesame 0x37a49f5 thru 0x37b067c data_size 0xc(0xbc88 bytes) 2024-09-13T09:27:51.922 INFO:teuthology.orchestra.run.smithi063.stdout:5275 punch from 0x1ec8ef8 to 0x1ed0636, (0x773e bytes) 2024-09-13T09:27:51.922 INFO:teuthology.orchestra.run.smithi063.stdout:5276 punch from 0x5308fd2 to 0x5312a00, (0x9a2e bytes) 2024-09-13T09:27:51.922 INFO:teuthology.orchestra.run.smithi063.stdout:5279 punch from 0x8b89479 to 0x8b94bdf, (0xb766 bytes) 2024-09-13T09:27:51.922 INFO:teuthology.orchestra.run.smithi063.stdout:5280 writesame 0x6d9fd61 thru 0x6dadc83 data_size 0x21(0xdf23 bytes) 2024-09-13T09:27:51.928 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:51.922+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:51.939 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:51.939 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:51.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:157000 3.1a 3:5ae3feea:::rbd_data.11183cfd6e74.00000000000001ba:head [set-alloc-hint object_size 262144 write_size 262144,write 61440~3072 in=3072b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:51.935181+0000", 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0033354539999999998, 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033652040000000001, 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 157000 2024-09-13T09:27:51.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935181+0000", 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935181+0000", 2024-09-13T09:27:51.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935180+0000", 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935189+0000", 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8799999999999997e-06 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935205+0000", 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6308999999999999e-05 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935217+0000", 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1946999999999999e-05 2024-09-13T09:27:51.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935301+0000", 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4320999999999994e-05 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935553+0000", 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00025160800000000001 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.935831+0000", 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027816000000000001 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:51.937115+0000", 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012836429999999999 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:51.945 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-13T09:27:51.946 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-13T09:27:51.946 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:51.949 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:51.949 INFO:teuthology.orchestra.run.smithi063.stdout:5282 write 0x4fe141d thru 0x4fe6081 (0x4c65 bytes) 2024-09-13T09:27:51.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:5284 write 0x2fe0cfc thru 0x2feef80 (0xe285 bytes) 2024-09-13T09:27:51.997 INFO:teuthology.orchestra.run.smithi063.stdout:5285 write 0x440cf96 thru 0x440e5bf (0x162a bytes) 2024-09-13T09:27:52.009 INFO:teuthology.orchestra.run.smithi063.stdout:5286 read 0x2d73412 thru 0x2d81d14 (0xe903 bytes) 2024-09-13T09:27:52.027 INFO:teuthology.orchestra.run.smithi063.stdout:5287 trunc from 0x91d0c29 to 0xd2676a 2024-09-13T09:27:52.029 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:52.022+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:52.063 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:52.064 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:52.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:52.102 INFO:teuthology.orchestra.run.smithi063.stdout:5288 write 0xa098340 thru 0xa09ec8e (0x694f bytes) 2024-09-13T09:27:52.130 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:52.126+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:52.133 INFO:teuthology.orchestra.run.smithi063.stdout:5289 write 0x66ce52b thru 0x66d908a (0xab60 bytes) 2024-09-13T09:27:52.169 INFO:teuthology.orchestra.run.smithi063.stdout:5290 writesame 0xc9e3be thru 0xca63e3 data_size 0x2f(0x8026 bytes) 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:52.187 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490692827000004, 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:27:52.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:27:52.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490670312999995, 2024-09-13T09:27:52.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.196 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490654476000003, 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.197 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.201 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490457531999994, 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:27:52.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:27:52.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490433207999999, 2024-09-13T09:27:52.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:27:52.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490416748000001, 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:27:52.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.215 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490400726999994, 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:27:52.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:27:52.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490350394000004, 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:27:52.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:27:52.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.490246819000006, 2024-09-13T09:27:52.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:27:52.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:27:52.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480869022999997, 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:27:52.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:27:52.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480850359000002, 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:27:52.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480833868999994, 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:27:52.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480818084000006, 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:27:52.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:27:52.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480801963000005, 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:27:52.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:27:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480786058000007, 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:27:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480769589000005, 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:27:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480747737000002, 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:27:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480730922000006, 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:27:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480714625000005, 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:27:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:27:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.480699024000003, 2024-09-13T09:27:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.281 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:5295 writesame 0x531e3b8 thru 0x53236a6 data_size 0x15(0x52ef bytes) 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:5296 read 0x63e2ee6 thru 0x63ee6c3 (0xb7de bytes) 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:5297 read 0x73395e3 thru 0x7341aa1 (0x84bf bytes) 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:5298 writesame 0x2de584d thru 0x2df49a3 data_size 0x1(0xf157 bytes) 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:5299 read 0x15a9cfc thru 0x15a9ea7 (0x1ac bytes) 2024-09-13T09:27:52.283 INFO:teuthology.orchestra.run.smithi063.stdout:5300 write 0xe57b12a thru 0xe57b163 (0x3a bytes) 2024-09-13T09:27:52.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:52.284 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:52.278+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:52.285 INFO:teuthology.orchestra.run.smithi063.stdout:5301 read 0x96727e8 thru 0x967fe07 (0xd620 bytes) 2024-09-13T09:27:52.285 INFO:teuthology.orchestra.run.smithi063.stdout:5302 read 0x50dd3df thru 0x50e4b31 (0x7753 bytes) 2024-09-13T09:27:52.298 INFO:teuthology.orchestra.run.smithi063.stdout:5303 write 0xe9f2ece thru 0xe9f73c3 (0x44f6 bytes) 2024-09-13T09:27:52.331 INFO:teuthology.orchestra.run.smithi063.stdout:5304 punch from 0xc2b38e0 to 0xc2bc9b2, (0x90d2 bytes) 2024-09-13T09:27:52.331 INFO:teuthology.orchestra.run.smithi063.stdout:5305 writesame 0xe6b7b8c thru 0xe6bd7bd data_size 0x1(0x5c32 bytes) 2024-09-13T09:27:52.360 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.360 INFO:teuthology.orchestra.run.smithi063.stdout:5308 read 0x435c7b8 thru 0x4364ee3 (0x872c bytes) 2024-09-13T09:27:52.361 INFO:teuthology.orchestra.run.smithi063.stdout:5309 read 0x3c1ef28 thru 0x3c26b77 (0x7c50 bytes) 2024-09-13T09:27:52.367 INFO:teuthology.orchestra.run.smithi063.stdout:5310 trunc from 0xe9f73c4 to 0xb81bdcf 2024-09-13T09:27:52.382 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:52.378+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:52.398 INFO:teuthology.orchestra.run.smithi063.stdout:5311 write 0xb0dbdd7 thru 0xb0e1e8f (0x60b9 bytes) 2024-09-13T09:27:52.428 INFO:teuthology.orchestra.run.smithi063.stdout:5312 read 0x200de21 thru 0x2016cda (0x8eba bytes) 2024-09-13T09:27:52.444 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:52.444 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:52.444 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:52.446 INFO:teuthology.orchestra.run.smithi063.stdout:5313 write 0xe21e20d thru 0xe21ea14 (0x808 bytes) 2024-09-13T09:27:52.452 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:27:52.452 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:27:52.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:52.469 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:27:52.471 INFO:teuthology.orchestra.run.smithi063.stdout:5314 write 0xe711101 thru 0xe715d75 (0x4c75 bytes) 2024-09-13T09:27:52.501 INFO:teuthology.orchestra.run.smithi063.stdout:5315 write 0x14175c1 thru 0x1425dd1 (0xe811 bytes) 2024-09-13T09:27:52.501 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:52.496+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:52.524 INFO:teuthology.orchestra.run.smithi063.stdout:5318 punch from 0x2129030 to 0x212ac36, (0x1c06 bytes) 2024-09-13T09:27:52.525 INFO:teuthology.orchestra.run.smithi063.stdout:5319 punch from 0xa121fd0 to 0xa12b48f, (0x94bf bytes) 2024-09-13T09:27:52.525 INFO:teuthology.orchestra.run.smithi063.stdout:5320 read 0x25a3025 thru 0x25ac571 (0x954d bytes) 2024-09-13T09:27:52.525 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.525 INFO:teuthology.orchestra.run.smithi063.stdout:5322 write 0x83014a2 thru 0x8310095 (0xebf4 bytes) 2024-09-13T09:27:52.556 INFO:teuthology.orchestra.run.smithi063.stdout:5323 trunc from 0xe715d76 to 0xac3e79e 2024-09-13T09:27:52.572 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:52.572 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:52.572 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:52.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:52.584 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:52.578+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:52.585 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.585 INFO:teuthology.orchestra.run.smithi063.stdout:5325 read 0x80d9555 thru 0x80e78d1 (0xe37d bytes) 2024-09-13T09:27:52.586 INFO:teuthology.orchestra.run.smithi063.stdout:5326 writesame 0xa796272 thru 0xa799d16 data_size 0x1(0x3aa5 bytes) 2024-09-13T09:27:52.618 INFO:teuthology.orchestra.run.smithi063.stdout:5327 punch from 0x1ae5df1 to 0x1ae94ab, (0x36ba bytes) 2024-09-13T09:27:52.618 INFO:teuthology.orchestra.run.smithi063.stdout:5330 read 0x5b3c0c2 thru 0x5b3da01 (0x1940 bytes) 2024-09-13T09:27:52.618 INFO:teuthology.orchestra.run.smithi063.stdout:5331 read 0xa582685 thru 0xa58b309 (0x8c85 bytes) 2024-09-13T09:27:52.619 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.633 INFO:teuthology.orchestra.run.smithi063.stdout:5335 trunc from 0xac3e79e to 0x18d8fda 2024-09-13T09:27:52.685 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:52.678+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:52.690 INFO:teuthology.orchestra.run.smithi063.stdout:5336 trunc from 0x18d8fda to 0x5d044d1 2024-09-13T09:27:52.694 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:52.694 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:52.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:52.694 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:52.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80778 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333263826999996, 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049178950999999999, 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80778 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358577+0000", 2024-09-13T09:27:52.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358579+0000", 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1619999999999998e-06 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358585+0000", 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358589+0000", 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2139999999999998e-06 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407748+0000", 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049158967999999997 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407756+0000", 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5040000000000002e-06 2024-09-13T09:27:52.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80779 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333248368, 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049256447000000002, 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80779 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358593+0000", 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358592+0000", 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358595+0000", 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358600+0000", 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1950000000000002e-06 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358604+0000", 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407781+0000", 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049176182999999998 2024-09-13T09:27:52.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407849+0000", 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8471000000000002e-05 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80780 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333232288000005, 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049322392, 2024-09-13T09:27:52.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80780 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358609+0000", 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358608+0000", 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358611+0000", 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4739999999999999e-06 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358616+0000", 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358621+0000", 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3150000000000002e-06 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407924+0000", 2024-09-13T09:27:52.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049303140000000002 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.407931+0000", 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80781 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333216515000004, 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049428727999999998, 2024-09-13T09:27:52.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80781 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358625+0000", 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358624+0000", 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358632+0000", 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358638+0000", 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358642+0000", 2024-09-13T09:27:52.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408013+0000", 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049371138000000002 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408053+0000", 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9926999999999999e-05 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.711 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80782 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333194614000007, 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049544887000000003, 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80782 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358646+0000", 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358648+0000", 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.396e-06 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358654+0000", 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2429999999999996e-06 2024-09-13T09:27:52.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358658+0000", 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408182+0000", 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049523695999999999 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408191+0000", 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4979999999999995e-06 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80783 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333178727999993, 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049560633, 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80783 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358662+0000", 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358664+0000", 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5450000000000002e-06 2024-09-13T09:27:52.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358669+0000", 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1699999999999996e-06 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358674+0000", 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3200000000000001e-06 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408216+0000", 2024-09-13T09:27:52.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049542022999999998 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408223+0000", 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1520000000000003e-06 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80784 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:52.720 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333163412999994, 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049577665999999999, 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80784 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358678+0000", 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358677+0000", 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358680+0000", 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4569999999999999e-06 2024-09-13T09:27:52.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358685+0000", 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358690+0000", 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408248+0000", 2024-09-13T09:27:52.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049558633999999997 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408255+0000", 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0779999999999998e-06 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80785 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333147367999999, 2024-09-13T09:27:52.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049617702999999999, 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80785 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358694+0000", 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358693+0000", 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358696+0000", 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:27:52.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358701+0000", 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3750000000000002e-06 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358706+0000", 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7180000000000004e-06 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408304+0000", 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049598058 2024-09-13T09:27:52.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408311+0000", 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6960000000000005e-06 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80786 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333131644000005, 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049631715999999999, 2024-09-13T09:27:52.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80786 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358709+0000", 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358711+0000", 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0930000000000001e-06 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358716+0000", 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358721+0000", 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408334+0000", 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049613551999999998 2024-09-13T09:27:52.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408341+0000", 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80787 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333117044000005, 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049683525999999999, 2024-09-13T09:27:52.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80787 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358724+0000", 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358726+0000", 2024-09-13T09:27:52.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3470000000000001e-06 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358731+0000", 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1170000000000003e-06 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358736+0000", 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408395+0000", 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049659637 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408408+0000", 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2183e-05 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80788 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333101615999993, 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049698607999999998, 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80788 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358739+0000", 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358742+0000", 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358747+0000", 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3489999999999999e-06 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358751+0000", 2024-09-13T09:27:52.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408431+0000", 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049679729999999998 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408438+0000", 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8510000000000001e-06 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:80789 3.c 3.9f5ec92c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.333085729999993, 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710888000000002, 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 80789 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.741 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358755+0000", 2024-09-13T09:27:52.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358757+0000", 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358762+0000", 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2390000000000002e-06 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.358768+0000", 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4829999999999999e-06 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408459+0000", 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049691483000000002 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.408466+0000", 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8220000000000001e-06 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.744 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85176 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1039360~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.983958486999995, 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048923136999999998, 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85176 2024-09-13T09:27:52.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707883+0000", 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707882+0000", 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707884+0000", 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1509999999999998e-06 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707889+0000", 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:27:52.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.707893+0000", 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2339999999999996e-06 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756606+0000", 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048712826000000001 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756686+0000", 2024-09-13T09:27:52.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0127e-05 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756806+0000", 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119702 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85178 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1022976~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:52.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.983824119999994, 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049010236999999998, 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85178 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:52.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708017+0000", 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708016+0000", 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708020+0000", 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.455e-06 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708027+0000", 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7139999999999996e-06 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708033+0000", 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9190000000000002e-06 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756828+0000", 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048794289999999997 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.756917+0000", 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757027+0000", 2024-09-13T09:27:52.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011001300000000001 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85179 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 1038336~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.983803229000003, 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049212661999999997, 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85179 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708038+0000", 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708037+0000", 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708040+0000", 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708045+0000", 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.708050+0000", 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5199999999999999e-06 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757053+0000", 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049003684999999998 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757125+0000", 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1433000000000005e-05 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.757251+0000", 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012583599999999999 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:52.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.609753624, 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:52.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:27:52.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97994 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 505856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.608876064, 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048717950000000003, 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97994 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082965+0000", 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082967+0000", 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.804e-06 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082970+0000", 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0539999999999999e-06 2024-09-13T09:27:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082974+0000", 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131472+0000", 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048498564000000001 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131549+0000", 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6994000000000002e-05 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131683+0000", 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000133669 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97996 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 521216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:27:52.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.608836504000003, 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951480999999998, 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97996 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083005+0000", 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083004+0000", 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083007+0000", 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5339999999999998e-06 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083012+0000", 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.083017+0000", 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131705+0000", 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048687646000000001 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131786+0000", 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1192999999999993e-05 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.131956+0000", 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017010300000000001 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.386513033, 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:27:52.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.859299827000001, 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:27:52.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:27:52.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:52.784 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:52.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:52.787 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:52.782+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5339 read 0x47117d4 thru 0x471cf6f (0xb79c bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5341 read 0x53e3401 thru 0x53e6bc7 (0x37c7 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5345 read 0x2612c08 thru 0x261530d (0x2706 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5346 write 0x8ac6aa3 thru 0x8ac9999 (0x2ef7 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5347 read 0x9d08a8 thru 0x9d6fdd (0x6736 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5349 read 0x5328314 thru 0x5338227 (0xff14 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5350 writesame 0x87b5cee thru 0x87ba925 data_size 0x12(0x4c38 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5351 read 0x25a2af6 thru 0x25adc77 (0xb182 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5353 read 0x9bcf48 thru 0x9be1ba (0x1273 bytes) 2024-09-13T09:27:52.788 INFO:teuthology.orchestra.run.smithi063.stdout:5354 trunc from 0x8ac999a to 0x119b0db 2024-09-13T09:27:52.805 INFO:teuthology.orchestra.run.smithi063.stdout:5355 punch from 0x9ce80f to 0x9cf2c6, (0xab7 bytes) 2024-09-13T09:27:52.805 INFO:teuthology.orchestra.run.smithi063.stdout:5358 punch from 0xcbfc4 to 0xd89bb, (0xc9f7 bytes) 2024-09-13T09:27:52.815 INFO:teuthology.orchestra.run.smithi063.stdout:5359 trunc from 0x119b0db to 0x6e6d1d9 2024-09-13T09:27:52.825 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.825 INFO:teuthology.orchestra.run.smithi063.stdout:5361 read 0x348798b thru 0x3497331 (0xf9a7 bytes) 2024-09-13T09:27:52.826 INFO:teuthology.orchestra.run.smithi063.stdout:5362 read 0x387c0c8 thru 0x38826fc (0x6635 bytes) 2024-09-13T09:27:52.844 INFO:teuthology.orchestra.run.smithi063.stdout:5363 writesame 0xac00e8c thru 0xac019d0 data_size 0x5(0xb45 bytes) 2024-09-13T09:27:52.844 INFO:teuthology.orchestra.run.smithi063.stdout:5364 read 0x516d201 thru 0x517851a (0xb31a bytes) 2024-09-13T09:27:52.844 INFO:teuthology.orchestra.run.smithi063.stdout:5365 punch from 0x53ece35 to 0x53ef871, (0x2a3c bytes) 2024-09-13T09:27:52.844 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:52.861 INFO:teuthology.orchestra.run.smithi063.stdout:5367 trunc from 0xac019d1 to 0x1b908f 2024-09-13T09:27:52.886 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:52.882+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:52.933 INFO:teuthology.orchestra.run.smithi063.stdout:5368 writesame 0xaaedfa7 thru 0xaaf1118 data_size 0x1(0x3172 bytes) 2024-09-13T09:27:52.933 INFO:teuthology.orchestra.run.smithi063.stdout:5369 write 0x6520854 thru 0x65271fe (0x69ab bytes) 2024-09-13T09:27:52.964 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:52.965 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:52.965 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:52.965 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:52.970 INFO:teuthology.orchestra.run.smithi063.stdout:5370 punch from 0xa828b87 to 0xa83860f, (0xfa88 bytes) 2024-09-13T09:27:52.970 INFO:teuthology.orchestra.run.smithi063.stdout:5371 writesame 0x681189c thru 0x6814db3 data_size 0x4(0x3518 bytes) 2024-09-13T09:27:52.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:52.986 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:52.982+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:53.000 INFO:teuthology.orchestra.run.smithi063.stdout:5373 read 0x6ae4b6a thru 0x6ae5ffa (0x1491 bytes) 2024-09-13T09:27:53.000 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:53.000 INFO:teuthology.orchestra.run.smithi063.stdout:5375 punch from 0x3b28fc to 0x3bce19, (0xa51d bytes) 2024-09-13T09:27:53.000 INFO:teuthology.orchestra.run.smithi063.stdout:5376 punch from 0x56f2fd2 to 0x56fb0ed, (0x811b bytes) 2024-09-13T09:27:53.000 INFO:teuthology.orchestra.run.smithi063.stdout:5377 read 0x52fb207 thru 0x53089bd (0xd7b7 bytes) 2024-09-13T09:27:53.015 INFO:teuthology.orchestra.run.smithi063.stdout:5378 write 0xcdc2212 thru 0xcdd0e90 (0xec7f bytes) 2024-09-13T09:27:53.046 INFO:teuthology.orchestra.run.smithi063.stdout:5379 read 0x14d76c9 thru 0x14e23b0 (0xace8 bytes) 2024-09-13T09:27:53.055 INFO:teuthology.orchestra.run.smithi063.stdout:5380 trunc from 0xcdd0e91 to 0x826a916 2024-09-13T09:27:53.078 INFO:teuthology.orchestra.run.smithi063.stdout:5381 read 0x6da4504 thru 0x6dae8dc (0xa3d9 bytes) 2024-09-13T09:27:53.082 INFO:teuthology.orchestra.run.smithi063.stdout:5383 trunc from 0x826a916 to 0x6a38160 2024-09-13T09:27:53.083 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:53.084 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:53.084 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:53.084 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:53.084 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:53.087 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:53.080+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:53.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:53.094 INFO:teuthology.orchestra.run.smithi063.stdout:5384 write 0x34593eb thru 0x34615fe (0x8214 bytes) 2024-09-13T09:27:53.146 INFO:teuthology.orchestra.run.smithi063.stdout:5385 write 0xb288b5d thru 0xb297b8b (0xf02f bytes) 2024-09-13T09:27:53.179 INFO:teuthology.orchestra.run.smithi063.stdout:5387 write 0x53f07c3 thru 0x53fb170 (0xa9ae bytes) 2024-09-13T09:27:53.188 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:53.184+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:53.206 INFO:teuthology.orchestra.run.smithi063.stdout:5388 punch from 0x220bf05 to 0x221be3c, (0xff37 bytes) 2024-09-13T09:27:53.207 INFO:teuthology.orchestra.run.smithi063.stdout:5389 read 0x58f5124 thru 0x5902235 (0xd112 bytes) 2024-09-13T09:27:53.207 INFO:teuthology.orchestra.run.smithi063.stdout:5390 write 0x3b0bcba thru 0x3b1625e (0xa5a5 bytes) 2024-09-13T09:27:53.232 INFO:teuthology.orchestra.run.smithi063.stdout:5391 read 0x39bb0dd thru 0x39bcf6d (0x1e91 bytes) 2024-09-13T09:27:53.233 INFO:teuthology.orchestra.run.smithi063.stdout:5392 read 0x9eb59e1 thru 0x9ebb74d (0x5d6d bytes) 2024-09-13T09:27:53.233 INFO:teuthology.orchestra.run.smithi063.stdout:5395 write 0x8763de2 thru 0x876d2d0 (0x94ef bytes) 2024-09-13T09:27:53.261 INFO:teuthology.orchestra.run.smithi063.stdout:5397 write 0x2488996 thru 0x24977d1 (0xee3c bytes) 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 113.70298405200001, 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:53.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:53.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.892760789999997, 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.634017919000001, 2024-09-13T09:27:53.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:53.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.631815580999998, 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:53.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.631418980000007, 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:53.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.630543595999995, 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:53.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.630522612999997, 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:53.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.630472100999995, 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.297 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:53.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.630442242000001, 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.305 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.58010256, 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:53.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:53.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.580043982000007, 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:53.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:53.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.580021346999999, 2024-09-13T09:27:53.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:53.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.580004384999995, 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.319 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:53.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.579132654999995, 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:53.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.579100021000002, 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:53.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:53.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.579079621999995, 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.579037936999995, 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.578905276, 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:53.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:53.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:53.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.576554822999995, 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:53.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.349 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.576370827999995, 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:53.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.354 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:53.356 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:53.356 INFO:teuthology.orchestra.run.smithi063.stdout:5399 write 0x60e72d0 thru 0x60f1c38 (0xa969 bytes) 2024-09-13T09:27:53.356 INFO:teuthology.orchestra.run.smithi063.stdout:5400 read 0xaa7bc0c thru 0xaa85d20 (0xa115 bytes) 2024-09-13T09:27:53.356 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:53.356 INFO:teuthology.orchestra.run.smithi063.stdout:5409 write 0xa3f5f7a thru 0xa401ad0 (0xbb57 bytes) 2024-09-13T09:27:53.356 INFO:teuthology.orchestra.run.smithi063.stdout:5410 writesame 0x93dbe30 thru 0x93e0380 data_size 0x41(0x4551 bytes) 2024-09-13T09:27:53.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:53.357 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:53.350+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:53.384 INFO:teuthology.orchestra.run.smithi063.stdout:5411 write 0x91e1776 thru 0x91ec401 (0xac8c bytes) 2024-09-13T09:27:53.437 INFO:teuthology.orchestra.run.smithi063.stdout:5412 writesame 0xe8e2e0d thru 0xe8e947c data_size 0x2c(0x6670 bytes) 2024-09-13T09:27:53.437 INFO:teuthology.orchestra.run.smithi063.stdout:5414 write 0xcefb29e thru 0xcf0a0c9 (0xee2c bytes) 2024-09-13T09:27:53.456 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:53.450+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:53.471 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:53.471 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:53.471 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:53.471 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:27:53.474 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:27:53.475 INFO:teuthology.orchestra.run.smithi063.stdout:5415 read 0x2d9763e thru 0x2da4cbb (0xd67e bytes) 2024-09-13T09:27:53.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:53.488 INFO:teuthology.orchestra.run.smithi063.stdout:5417 trunc from 0xe8e947d to 0x86d4e24 2024-09-13T09:27:53.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:27:53.549 INFO:teuthology.orchestra.run.smithi063.stdout:5418 punch from 0x55990f6 to 0x55a0df5, (0x7cff bytes) 2024-09-13T09:27:53.549 INFO:teuthology.orchestra.run.smithi063.stdout:5419 punch from 0x57d583a to 0x57e1946, (0xc10c bytes) 2024-09-13T09:27:53.549 INFO:teuthology.orchestra.run.smithi063.stdout:5423 writesame 0x6c5622b thru 0x6c5a2cc data_size 0x1(0x40a2 bytes) 2024-09-13T09:27:53.557 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:53.552+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:53.588 INFO:teuthology.orchestra.run.smithi063.stdout:5425 trunc from 0x86d4e24 to 0x45e0b21 2024-09-13T09:27:53.599 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:53.599 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:53.599 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:53.599 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:53.600 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:53.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:53.637 INFO:teuthology.orchestra.run.smithi063.stdout:5428 write 0x22020ab thru 0x2202397 (0x2ed bytes) 2024-09-13T09:27:53.658 INFO:teuthology.orchestra.run.smithi063.stdout:5429 write 0x983c8c1 thru 0x9849845 (0xcf85 bytes) 2024-09-13T09:27:53.658 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:53.652+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:53.690 INFO:teuthology.orchestra.run.smithi063.stdout:5432 punch from 0x743fef1 to 0x744b401, (0xb510 bytes) 2024-09-13T09:27:53.690 INFO:teuthology.orchestra.run.smithi063.stdout:5434 read 0x47073ad thru 0x470a8a4 (0x34f8 bytes) 2024-09-13T09:27:53.690 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:53.690 INFO:teuthology.orchestra.run.smithi063.stdout:5437 punch from 0x35c31a8 to 0x35c70bd, (0x3f15 bytes) 2024-09-13T09:27:53.690 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:53.706 INFO:teuthology.orchestra.run.smithi063.stdout:5439 writesame 0xe4ba196 thru 0xe4c1c6f data_size 0x25(0x7ada bytes) 2024-09-13T09:27:53.710 INFO:teuthology.orchestra.run.smithi063.stdout:5440 trunc from 0xe4c1c70 to 0xc927598 2024-09-13T09:27:53.720 INFO:teuthology.orchestra.run.smithi063.stdout:5442 writesame 0x8eb7b77 thru 0x8ec08f9 data_size 0x1(0x8d83 bytes) 2024-09-13T09:27:53.725 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:53.725 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:53.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:53.725 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:53.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 130.761361034, 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:53.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.730 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 67.648141289999998, 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:53.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 67.647743274999996, 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:53.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972636398999995, 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.740 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:53.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:53.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972621466999996, 2024-09-13T09:27:53.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972588361000007, 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972572068000005, 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:53.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:53.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972555952999997, 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972524911999997, 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:53.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:53.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.972493288999999, 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:53.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959545227999996, 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:53.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:53.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959528172000006, 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:53.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:53.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:53.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959512004000004, 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:53.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:53.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959298341999997, 2024-09-13T09:27:53.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:53.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959279030999994, 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:53.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959262637999998, 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:53.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959246976000003, 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:53.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.959220622000004, 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:53.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:53.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 20.430707895000001, 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:27:53.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:53.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.31879378, 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:53.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:27:53.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:53.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:53.821 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:53.822 INFO:teuthology.orchestra.run.smithi063.stdout:5443 write 0xdb2dc71 thru 0xdb3bcb8 (0xe048 bytes) 2024-09-13T09:27:53.822 INFO:teuthology.orchestra.run.smithi063.stdout:5444 read 0x1e0ff5d thru 0x1e17c8f (0x7d33 bytes) 2024-09-13T09:27:53.822 INFO:teuthology.orchestra.run.smithi063.stdout:5447 writesame 0x9c668e6 thru 0x9c759d1 data_size 0x22(0xf0ec bytes) 2024-09-13T09:27:53.822 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:53.816+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:53.823 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:53.844 INFO:teuthology.orchestra.run.smithi063.stdout:5448 write 0x2a7782b thru 0x2a78d87 (0x155d bytes) 2024-09-13T09:27:53.875 INFO:teuthology.orchestra.run.smithi063.stdout:5449 trunc from 0xdb3bcb9 to 0x2c59ab0 2024-09-13T09:27:53.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:53.936 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:53.932+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:53.944 INFO:teuthology.orchestra.run.smithi063.stdout:5450 write 0xc70214 thru 0xc7ba40 (0xb82d bytes) 2024-09-13T09:27:53.948 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159022 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944277+0000", 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0026620860000000001, 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002677558, 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.949 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159022 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944277+0000", 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944277+0000", 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944277+0000", 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944280+0000", 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8380000000000002e-06 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944284+0000", 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3329999999999998e-06 2024-09-13T09:27:53.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944288+0000", 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0099999999999997e-06 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159023 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944305+0000", 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.002634506, 2024-09-13T09:27:53.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002683497, 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159023 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944305+0000", 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944305+0000", 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944305+0000", 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944308+0000", 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7479999999999999e-06 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944315+0000", 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944319+0000", 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2560000000000004e-06 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159024 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944323+0000", 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0026161420000000001, 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0026917, 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159024 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944323+0000", 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944323+0000", 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944323+0000", 2024-09-13T09:27:53.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944326+0000", 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8700000000000001e-06 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944331+0000", 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5230000000000003e-06 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944335+0000", 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8970000000000001e-06 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159010 3.2 3:4420c587:::rbd_object_map.11183cfd6e74:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.959 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.943834+0000", 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0031056320000000001, 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0032129419999999999, 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159010 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943834+0000", 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943834+0000", 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943832+0000", 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943843+0000", 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1287000000000001e-05 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943867+0000", 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3819999999999999e-05 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943879+0000", 2024-09-13T09:27:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1527e-05 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943966+0000", 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7404999999999994e-05 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944117+0000", 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015086900000000001 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944523+0000", 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00040600500000000003 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159011 3.2 3:4420c587:::rbd_object_map.11183cfd6e74:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.943925+0000", 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0030149209999999998, 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031611550000000001, 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159011 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943925+0000", 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943925+0000", 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943924+0000", 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943932+0000", 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1310000000000006e-06 2024-09-13T09:27:53.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943952+0000", 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0564000000000002e-05 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943962+0000", 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3919999999999992e-06 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.945723+0000", 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0017610869999999999 2024-09-13T09:27:53.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.945844+0000", 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012174499999999999 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.946307+0000", 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000462888 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159012 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.943975+0000", 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0029645159999999999, 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00314445, 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159012 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.969 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943975+0000", 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943975+0000", 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943974+0000", 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943980+0000", 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943990+0000", 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7520000000000008e-06 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.943997+0000", 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3629999999999996e-06 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159013 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944017+0000", 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0029228879999999998, 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031290990000000002, 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159013 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944017+0000", 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944017+0000", 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944016+0000", 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944021+0000", 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3650000000000003e-06 2024-09-13T09:27:53.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944053+0000", 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1312000000000001e-05 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944062+0000", 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0000000000000002e-06 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159014 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944070+0000", 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0028698209999999998, 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031039169999999999, 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159014 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944070+0000", 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944070+0000", 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944069+0000", 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944074+0000", 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2650000000000004e-06 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944102+0000", 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7846000000000001e-05 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944114+0000", 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2197e-05 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159015 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944148+0000", 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.002791385, 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030535739999999999, 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159015 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944148+0000", 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944148+0000", 2024-09-13T09:27:53.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944147+0000", 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944154+0000", 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4400000000000002e-06 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944165+0000", 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1482e-05 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944173+0000", 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9149999999999996e-06 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159016 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.982 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944178+0000", 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.002761173, 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030577299999999998, 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159016 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944178+0000", 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944178+0000", 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944178+0000", 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944181+0000", 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944195+0000", 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3334e-05 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944200+0000", 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:27:53.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159017 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944204+0000", 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0027352119999999999, 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030579959999999999, 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159017 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944204+0000", 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944204+0000", 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944204+0000", 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944207+0000", 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1149999999999998e-06 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944212+0000", 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6979999999999997e-06 2024-09-13T09:27:53.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944216+0000", 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159018 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944219+0000", 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0027205350000000001, 2024-09-13T09:27:53.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030688529999999999, 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159018 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944219+0000", 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944219+0000", 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944219+0000", 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944221+0000", 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:27:53.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944226+0000", 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.374e-06 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944229+0000", 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7069999999999998e-06 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159019 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944233+0000", 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0027067649999999999, 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0030808799999999998, 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159019 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.993 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944233+0000", 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944233+0000", 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944232+0000", 2024-09-13T09:27:53.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944235+0000", 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944240+0000", 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2689999999999999e-06 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944244+0000", 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.053e-06 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159020 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944248+0000", 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0026916570000000001, 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00309159, 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:53.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159020 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944248+0000", 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:53.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944248+0000", 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944247+0000", 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944251+0000", 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1949999999999999e-06 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944255+0000", 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4320000000000001e-06 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944260+0000", 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8600000000000001e-06 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:53.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:159021 3.2 3.e1a30422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.944264+0000", 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.0026759150000000001, 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0031015909999999999, 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "queued for pg", 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 159021 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:54.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944264+0000", 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944264+0000", 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944263+0000", 2024-09-13T09:27:54.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944266+0000", 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8049999999999998e-06 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944270+0000", 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4839999999999998e-06 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:54.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.944274+0000", 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6200000000000001e-06 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 15 2024-09-13T09:27:54.003 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:54.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:54.005 INFO:teuthology.orchestra.run.smithi063.stdout:5453 read 0xf53343 thru 0xf538a1 (0x55f bytes) 2024-09-13T09:27:54.005 INFO:teuthology.orchestra.run.smithi063.stdout:5454 writesame 0x8589238 thru 0x85903a3 data_size 0x11(0x716c bytes) 2024-09-13T09:27:54.006 INFO:teuthology.orchestra.run.smithi063.stdout:5456 read 0x24ed477 thru 0x24fc718 (0xf2a2 bytes) 2024-09-13T09:27:54.006 INFO:teuthology.orchestra.run.smithi063.stdout:5458 punch from 0xd9964 to 0xdcde3, (0x347f bytes) 2024-09-13T09:27:54.006 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:54.006 INFO:teuthology.orchestra.run.smithi063.stdout:5460 write 0x5e321a6 thru 0x5e3c41b (0xa276 bytes) 2024-09-13T09:27:54.024 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:54.018+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:54.034 INFO:teuthology.orchestra.run.smithi063.stdout:5461 punch from 0x833b1ad to 0x833efc2, (0x3e15 bytes) 2024-09-13T09:27:54.044 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:54.056 INFO:teuthology.orchestra.run.smithi063.stdout:5462 writesame 0xedfaa74 thru 0xee05049 data_size 0x1(0xa5d6 bytes) 2024-09-13T09:27:54.056 INFO:teuthology.orchestra.run.smithi063.stdout:5463 writesame 0x199a97d thru 0x19a561a data_size 0x2d(0xac9e bytes) 2024-09-13T09:27:54.073 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:54.089 INFO:teuthology.orchestra.run.smithi063.stdout:5464 write 0x3b6c85d thru 0x3b7527d (0x8a21 bytes) 2024-09-13T09:27:54.100 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:54.116 INFO:teuthology.orchestra.run.smithi063.stdout:5466 trunc from 0xee0504a to 0xea9e1ad 2024-09-13T09:27:54.122 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:54.122 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:54.122 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:54.122 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:54.123 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:54.125 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:54.125 INFO:teuthology.orchestra.run.smithi063.stdout:5468 write 0xac6fc3e thru 0xac70e07 (0x11ca bytes) 2024-09-13T09:27:54.125 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:54.122+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:54.135 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:54.138 INFO:teuthology.orchestra.run.smithi063.stdout:5470 write 0x6bacb83 thru 0x6bb88ca (0xbd48 bytes) 2024-09-13T09:27:54.150 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:54.169 INFO:teuthology.orchestra.run.smithi063.stdout:5471 write 0xe153e90 thru 0xe15d32f (0x94a0 bytes) 2024-09-13T09:27:54.189 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:54.207 INFO:teuthology.orchestra.run.smithi063.stdout:5472 trunc from 0xea9e1ad to 0x8e2ff10 2024-09-13T09:27:54.226 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:54.226 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:54.220+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:54.239 INFO:teuthology.orchestra.run.smithi063.stdout:5473 punch from 0x1290dd5 to 0x129dd0b, (0xcf36 bytes) 2024-09-13T09:27:54.239 INFO:teuthology.orchestra.run.smithi063.stdout:5475 punch from 0xfe3836 to 0xfe4c02, (0x13cc bytes) 2024-09-13T09:27:54.239 INFO:teuthology.orchestra.run.smithi063.stdout:5477 write 0x6624a79 thru 0x66294eb (0x4a73 bytes) 2024-09-13T09:27:54.260 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:54.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:54.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:54.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:54.263 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:54.277 INFO:teuthology.orchestra.run.smithi063.stdout:5479 punch from 0x3d22d14 to 0x3d2b85d, (0x8b49 bytes) 2024-09-13T09:27:54.278 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:54.278 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:54.281 INFO:teuthology.orchestra.run.smithi063.stdout:5480 trunc from 0x8e2ff10 to 0xba332ec 2024-09-13T09:27:54.289 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:54.296 INFO:teuthology.orchestra.run.smithi063.stdout:5481 trunc from 0xba332ec to 0xe6c8444 2024-09-13T09:27:54.310 INFO:teuthology.orchestra.run.smithi063.stdout:5483 writesame 0xcaad8a4 thru 0xcab0981 data_size 0x9(0x30de bytes) 2024-09-13T09:27:54.327 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:54.320+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:54.342 INFO:teuthology.orchestra.run.smithi063.stdout:5484 punch from 0xa814c92 to 0xa8164db, (0x1849 bytes) 2024-09-13T09:27:54.347 INFO:teuthology.orchestra.run.smithi063.stdout:5485 trunc from 0xe6c8444 to 0xb7ad8d1 2024-09-13T09:27:54.374 INFO:teuthology.orchestra.run.smithi063.stdout:5486 read 0x9bd03d thru 0x9c2409 (0x53cd bytes) 2024-09-13T09:27:54.374 INFO:teuthology.orchestra.run.smithi063.stdout:5488 write 0x63b3ec8 thru 0x63b6ec9 (0x3002 bytes) 2024-09-13T09:27:54.404 INFO:teuthology.orchestra.run.smithi063.stdout:5490 punch from 0xa494aa6 to 0xa49e4ac, (0x9a06 bytes) 2024-09-13T09:27:54.404 INFO:teuthology.orchestra.run.smithi063.stdout:5492 read 0x5500330 thru 0x550483f (0x4510 bytes) 2024-09-13T09:27:54.408 INFO:teuthology.orchestra.run.smithi063.stdout:5493 trunc from 0xb7ad8d1 to 0x8926116 2024-09-13T09:27:54.418 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:54.418 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:54.419 INFO:teuthology.orchestra.run.smithi063.stdout:5494 write 0x4e5f171 thru 0x4e62ae9 (0x3979 bytes) 2024-09-13T09:27:54.427 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:54.420+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:54.428 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:54.449 INFO:teuthology.orchestra.run.smithi063.stdout:5495 writesame 0xf4ac9b thru 0xf4b061 data_size 0x1(0x3c7 bytes) 2024-09-13T09:27:54.456 INFO:teuthology.orchestra.run.smithi063.stdout:5496 read 0x7958737 thru 0x79613c1 (0x8c8b bytes) 2024-09-13T09:27:54.475 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:27:54.477 INFO:teuthology.orchestra.run.smithi063.stdout:5497 writesame 0xcd4fc56 thru 0xcd502a1 data_size 0x1(0x64c bytes) 2024-09-13T09:27:54.478 INFO:teuthology.orchestra.run.smithi063.stdout:5498 writesame 0x6b8e52d thru 0x6b9dac4 data_size 0x74(0xf598 bytes) 2024-09-13T09:27:54.492 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:27:54.493 INFO:teuthology.orchestra.run.smithi063.stdout:5499 read 0x98b6446 thru 0x98bfca2 (0x985d bytes) 2024-09-13T09:27:54.493 INFO:teuthology.orchestra.run.smithi063.stdout:5500 read 0x3fa7a34 thru 0x3fb09d1 (0x8f9e bytes) 2024-09-13T09:27:54.502 INFO:teuthology.orchestra.run.smithi063.stdout:5502 write 0xd3811c8 thru 0xd38b317 (0xa150 bytes) 2024-09-13T09:27:54.528 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:54.524+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:54.538 INFO:teuthology.orchestra.run.smithi063.stdout:5503 read 0x2edf44b thru 0x2ee9278 (0x9e2e bytes) 2024-09-13T09:27:54.538 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:54.539 INFO:teuthology.orchestra.run.smithi063.stdout:5505 write 0x639a229 thru 0x639d480 (0x3258 bytes) 2024-09-13T09:27:54.549 INFO:teuthology.orchestra.run.smithi063.stdout:5506 write 0x7e78aa9 thru 0x7e7acb9 (0x2211 bytes) 2024-09-13T09:27:54.571 INFO:teuthology.orchestra.run.smithi063.stdout:5507 punch from 0x1b4adb8 to 0x1b4ddb3, (0x2ffb bytes) 2024-09-13T09:27:54.571 INFO:teuthology.orchestra.run.smithi063.stdout:5508 writesame 0x2814ec0 thru 0x281db46 data_size 0x5(0x8c87 bytes) 2024-09-13T09:27:54.582 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:54.582 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:54.595 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:54.603 INFO:teuthology.orchestra.run.smithi063.stdout:5509 writesame 0x63b85d9 thru 0x63c457b data_size 0x4f(0xbfa3 bytes) 2024-09-13T09:27:54.612 INFO:teuthology.orchestra.run.smithi063.stdout:5510 writesame 0xa8323df thru 0xa838b5e data_size 0x60(0x6780 bytes) 2024-09-13T09:27:54.629 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:54.624+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:54.640 INFO:teuthology.orchestra.run.smithi063.stdout:5511 writesame 0x3ee7a5f thru 0x3eef0bf data_size 0x5(0x7661 bytes) 2024-09-13T09:27:54.666 INFO:teuthology.orchestra.run.smithi063.stdout:5512 write 0x58506d thru 0x58c670 (0x7604 bytes) 2024-09-13T09:27:54.718 INFO:teuthology.orchestra.run.smithi063.stdout:5513 trunc from 0xd38b318 to 0x228718d 2024-09-13T09:27:54.730 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:54.724+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:54.739 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:54.739 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:54.753 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:54.824 INFO:teuthology.orchestra.run.smithi063.stdout:5514 read 0x1c3f26b thru 0x1c4c58c (0xd322 bytes) 2024-09-13T09:27:54.832 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:54.826+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:54.842 INFO:teuthology.orchestra.run.smithi063.stdout:5515 writesame 0x692799b thru 0x692a743 data_size 0x1(0x2da9 bytes) 2024-09-13T09:27:54.847 INFO:teuthology.orchestra.run.smithi063.stdout:5517 trunc from 0x692a744 to 0x3e97eb2 2024-09-13T09:27:54.860 INFO:teuthology.orchestra.run.smithi063.stdout:5520 read 0x26096d thru 0x261d8f (0x1423 bytes) 2024-09-13T09:27:54.868 INFO:teuthology.orchestra.run.smithi063.stdout:5521 trunc from 0x3e97eb2 to 0x905037a 2024-09-13T09:27:54.883 INFO:teuthology.orchestra.run.smithi063.stdout:5522 writesame 0x82e5ac0 thru 0x82f4aad data_size 0x57(0xefee bytes) 2024-09-13T09:27:54.916 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:54.916 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:54.922 INFO:teuthology.orchestra.run.smithi063.stdout:5523 writesame 0x26e290e thru 0x26f006a data_size 0x1(0xd75d bytes) 2024-09-13T09:27:54.927 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:54.932 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:54.926+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:54.954 INFO:teuthology.orchestra.run.smithi063.stdout:5525 punch from 0x45759ae to 0x4581709, (0xbd5b bytes) 2024-09-13T09:27:54.966 INFO:teuthology.orchestra.run.smithi063.stdout:5526 writesame 0xa4c3397 thru 0xa4c670c data_size 0x7(0x3376 bytes) 2024-09-13T09:27:54.980 INFO:teuthology.orchestra.run.smithi063.stdout:5528 trunc from 0xa4c670d to 0x1937f39 2024-09-13T09:27:55.028 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:55.033 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:55.026+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:55.037 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.037 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.049 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:55.055 INFO:teuthology.orchestra.run.smithi063.stdout:5532 writesame 0x9f1c60a thru 0x9f1f5b8 data_size 0xd(0x2faf bytes) 2024-09-13T09:27:55.055 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:55.055 INFO:teuthology.orchestra.run.smithi063.stdout:5535 read 0x6916e8b thru 0x6923166 (0xc2dc bytes) 2024-09-13T09:27:55.069 INFO:teuthology.orchestra.run.smithi063.stdout:5536 write 0xa8624ce thru 0xa87176f (0xf2a2 bytes) 2024-09-13T09:27:55.109 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:55.109 INFO:teuthology.orchestra.run.smithi063.stdout:5538 read 0x90bfe3d thru 0x90c4947 (0x4b0b bytes) 2024-09-13T09:27:55.110 INFO:teuthology.orchestra.run.smithi063.stdout:5539 punch from 0x7dc57e3 to 0x7dca241, (0x4a5e bytes) 2024-09-13T09:27:55.110 INFO:teuthology.orchestra.run.smithi063.stdout:5540 read 0x42ecca1 thru 0x42ef729 (0x2a89 bytes) 2024-09-13T09:27:55.110 INFO:teuthology.orchestra.run.smithi063.stdout:5541 read 0x47f2ba thru 0x486c3b (0x7982 bytes) 2024-09-13T09:27:55.131 INFO:teuthology.orchestra.run.smithi063.stdout:5542 write 0xee148bc thru 0xee1e0f6 (0x983b bytes) 2024-09-13T09:27:55.133 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:55.130+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:55.163 INFO:teuthology.orchestra.run.smithi063.stdout:5545 read 0x3fa0a34 thru 0x3fa74b0 (0x6a7d bytes) 2024-09-13T09:27:55.165 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.166 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.174 INFO:teuthology.orchestra.run.smithi063.stdout:5547 trunc from 0xee1e0f7 to 0x7e27ec0 2024-09-13T09:27:55.179 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:55.234 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:55.228+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:55.236 INFO:teuthology.orchestra.run.smithi063.stdout:5550 write 0xd0427c4 thru 0xd042e06 (0x643 bytes) 2024-09-13T09:27:55.248 INFO:teuthology.orchestra.run.smithi063.stdout:5551 punch from 0x2bfb682 to 0x2c05687, (0xa005 bytes) 2024-09-13T09:27:55.248 INFO:teuthology.orchestra.run.smithi063.stdout:5552 write 0xbcbe3f1 thru 0xbcca867 (0xc477 bytes) 2024-09-13T09:27:55.284 INFO:teuthology.orchestra.run.smithi063.stdout:5553 writesame 0x713d150 thru 0x71498fa data_size 0x1(0xc7ab bytes) 2024-09-13T09:27:55.316 INFO:teuthology.orchestra.run.smithi063.stdout:5554 write 0xc800521 thru 0xc8032b6 (0x2d96 bytes) 2024-09-13T09:27:55.335 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:55.328+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:55.352 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.353 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.361 INFO:teuthology.orchestra.run.smithi063.stdout:5555 write 0xe789592 thru 0xe794bca (0xb639 bytes) 2024-09-13T09:27:55.363 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:55.397 INFO:teuthology.orchestra.run.smithi063.stdout:5556 read 0x9bb491f thru 0x9bb8854 (0x3f36 bytes) 2024-09-13T09:27:55.397 INFO:teuthology.orchestra.run.smithi063.stdout:5557 write 0x564bdd7 thru 0x564e53b (0x2765 bytes) 2024-09-13T09:27:55.440 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:55.436+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:55.442 INFO:teuthology.orchestra.run.smithi063.stdout:5559 trunc from 0xe794bcb to 0x626ecfb 2024-09-13T09:27:55.493 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:27:55.501 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:27:55.503 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:27:55.521 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.521 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.525 INFO:teuthology.orchestra.run.smithi063.stdout:5560 writesame 0xcfc8157 thru 0xcfd4926 data_size 0x5c(0xc7d0 bytes) 2024-09-13T09:27:55.525 INFO:teuthology.orchestra.run.smithi063.stdout:5561 write 0x9bc4e6a thru 0x9bca9a1 (0x5b38 bytes) 2024-09-13T09:27:55.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:55.537 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:55.532+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:5562 trunc from 0xcfd4927 to 0x4778c2a 2024-09-13T09:27:55.614 INFO:teuthology.orchestra.run.smithi063.stdout:5563 read 0x57acf8 thru 0x587986 (0xcc8f bytes) 2024-09-13T09:27:55.616 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:55.633 INFO:teuthology.orchestra.run.smithi063.stdout:5565 writesame 0x86432a5 thru 0x864a6ed data_size 0x1(0x7449 bytes) 2024-09-13T09:27:55.633 INFO:teuthology.orchestra.run.smithi063.stdout:5566 read 0x64d02a6 thru 0x64d1030 (0xd8b bytes) 2024-09-13T09:27:55.633 INFO:teuthology.orchestra.run.smithi063.stdout:5568 punch from 0x69213f8 to 0x692302b, (0x1c33 bytes) 2024-09-13T09:27:55.633 INFO:teuthology.orchestra.run.smithi063.stdout:5569 read 0xc26019 thru 0xc2c3fb (0x63e3 bytes) 2024-09-13T09:27:55.637 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:55.634+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:55.657 INFO:teuthology.orchestra.run.smithi063.stdout:5572 write 0xe5405b9 thru 0xe5413f3 (0xe3b bytes) 2024-09-13T09:27:55.660 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.660 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.667 INFO:teuthology.orchestra.run.smithi063.stdout:5573 writesame 0x15dc047 thru 0x15e018c data_size 0xf(0x4146 bytes) 2024-09-13T09:27:55.676 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:55.704 INFO:teuthology.orchestra.run.smithi063.stdout:5574 read 0xc6e5702 thru 0xc6f11a5 (0xbaa4 bytes) 2024-09-13T09:27:55.704 INFO:teuthology.orchestra.run.smithi063.stdout:5578 punch from 0x3e19c0f to 0x3e1f038, (0x5429 bytes) 2024-09-13T09:27:55.704 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:55.725 INFO:teuthology.orchestra.run.smithi063.stdout:5584 trunc from 0xe5413f4 to 0x1384f80 2024-09-13T09:27:55.737 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:55.734+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:55.760 INFO:teuthology.orchestra.run.smithi063.stdout:5585 read 0x1302801 thru 0x130b380 (0x8b80 bytes) 2024-09-13T09:27:55.761 INFO:teuthology.orchestra.run.smithi063.stdout:5586 punch from 0x907657 to 0x916a61, (0xf40a bytes) 2024-09-13T09:27:55.761 INFO:teuthology.orchestra.run.smithi063.stdout:5587 punch from 0x10abfdb to 0x10b032f, (0x4354 bytes) 2024-09-13T09:27:55.775 INFO:teuthology.orchestra.run.smithi063.stdout:5590 write 0x32d76ff thru 0x32e675c (0xf05e bytes) 2024-09-13T09:27:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:55.834 INFO:teuthology.orchestra.run.smithi063.stdout:5591 write 0xb8af9d8 thru 0xb8b12b7 (0x18e0 bytes) 2024-09-13T09:27:55.839 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:55.832+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:55.855 INFO:teuthology.orchestra.run.smithi063.stdout:5592 write 0x5cc16f0 thru 0x5ccaad3 (0x93e4 bytes) 2024-09-13T09:27:55.893 INFO:teuthology.orchestra.run.smithi063.stdout:5593 write 0x5419ef thru 0x54c10c (0xa71e bytes) 2024-09-13T09:27:55.900 INFO:teuthology.orchestra.run.smithi063.stdout:5594 writesame 0xaee94aa thru 0xaef1216 data_size 0x4d(0x7d6d bytes) 2024-09-13T09:27:55.908 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:55.908 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:55.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:55.935 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:55.935 INFO:teuthology.orchestra.run.smithi063.stdout:5596 write 0xac978c5 thru 0xaca0c80 (0x93bc bytes) 2024-09-13T09:27:55.940 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:55.934+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:55.943 INFO:teuthology.orchestra.run.smithi063.stdout:5597 read 0x173c3d2 thru 0x1749f00 (0xdb2f bytes) 2024-09-13T09:27:55.943 INFO:teuthology.orchestra.run.smithi063.stdout:5598 write 0x16f4 thru 0x653b (0x4e48 bytes) 2024-09-13T09:27:55.953 INFO:teuthology.orchestra.run.smithi063.stdout:5600 trunc from 0xb8b12b8 to 0xc2f79ef 2024-09-13T09:27:55.961 INFO:teuthology.orchestra.run.smithi063.stdout:5601 read 0x6fb9c3 thru 0x70aae1 (0xf11f bytes) 2024-09-13T09:27:55.962 INFO:teuthology.orchestra.run.smithi063.stdout:5602 read 0x4356694 thru 0x43575f7 (0xf64 bytes) 2024-09-13T09:27:55.963 INFO:teuthology.orchestra.run.smithi063.stdout:5603 write 0x974e3e5 thru 0x9757561 (0x917d bytes) 2024-09-13T09:27:56.001 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:56.025 INFO:teuthology.orchestra.run.smithi063.stdout:5605 trunc from 0xc2f79ef to 0xc98b2 2024-09-13T09:27:56.026 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:56.026 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:56.039 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:56.082 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:56.078+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:56.110 INFO:teuthology.orchestra.run.smithi063.stdout:5607 read 0xadef7 thru 0xbb904 (0xda0e bytes) 2024-09-13T09:27:56.122 INFO:teuthology.orchestra.run.smithi063.stdout:5608 trunc from 0xc98b2 to 0x40a32ea 2024-09-13T09:27:56.134 INFO:teuthology.orchestra.run.smithi063.stdout:5609 read 0x16b445c thru 0x16b60b1 (0x1c56 bytes) 2024-09-13T09:27:56.141 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:56.136+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:56.143 INFO:teuthology.orchestra.run.smithi063.stdout:5610 trunc from 0x40a32ea to 0x83d7d68 2024-09-13T09:27:56.157 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:56.157 INFO:teuthology.orchestra.run.smithi063.stdout:5612 read 0x301eaf8 thru 0x3027f62 (0x946b bytes) 2024-09-13T09:27:56.158 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:56.158 INFO:teuthology.orchestra.run.smithi063.stdout:5614 read 0x77377ef thru 0x773e9ab (0x71bd bytes) 2024-09-13T09:27:56.158 INFO:teuthology.orchestra.run.smithi063.stdout:5615 read 0x18a1ff8 thru 0x18adc90 (0xbc99 bytes) 2024-09-13T09:27:56.169 INFO:teuthology.orchestra.run.smithi063.stdout:5616 trunc from 0x83d7d68 to 0xda5649b 2024-09-13T09:27:56.182 INFO:teuthology.orchestra.run.smithi063.stdout:5617 write 0x5215cf thru 0x528ec3 (0x78f5 bytes) 2024-09-13T09:27:56.185 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:56.186 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:56.196 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:56.220 INFO:teuthology.orchestra.run.smithi063.stdout:5618 read 0x98adbf6 thru 0x98afd3f (0x214a bytes) 2024-09-13T09:27:56.221 INFO:teuthology.orchestra.run.smithi063.stdout:5620 write 0x30c8e7b thru 0x30cb0ca (0x2250 bytes) 2024-09-13T09:27:56.242 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:56.236+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:56.251 INFO:teuthology.orchestra.run.smithi063.stdout:5621 trunc from 0xda5649b to 0xbca93e0 2024-09-13T09:27:56.263 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:56.263 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:56.263 INFO:teuthology.orchestra.run.smithi063.stdout:5626 write 0x283052c thru 0x2838aed (0x85c2 bytes) 2024-09-13T09:27:56.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:27:56.298 INFO:teuthology.orchestra.run.smithi063.stdout:5627 read 0x8b1385e thru 0x8b19a9d (0x6240 bytes) 2024-09-13T09:27:56.299 INFO:teuthology.orchestra.run.smithi063.stdout:5629 punch from 0x7d27f03 to 0x7d2d854, (0x5951 bytes) 2024-09-13T09:27:56.301 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:56.302 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:56.309 INFO:teuthology.orchestra.run.smithi063.stdout:5631 trunc from 0xbca93e0 to 0x88617f2 2024-09-13T09:27:56.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:27:56.336 INFO:teuthology.orchestra.run.smithi063.stdout:5632 writesame 0xc60908f thru 0xc60dfca data_size 0x16(0x4f3c bytes) 2024-09-13T09:27:56.344 INFO:teuthology.orchestra.run.smithi063.stdout:5633 trunc from 0xc60dfcb to 0x7eda7f0 2024-09-13T09:27:56.357 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:56.354+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:56.358 INFO:teuthology.orchestra.run.smithi063.stdout:5634 read 0x44e2c50 thru 0x44e73d2 (0x4783 bytes) 2024-09-13T09:27:56.358 INFO:teuthology.orchestra.run.smithi063.stdout:5635 punch from 0x425f44a to 0x4269a97, (0xa64d bytes) 2024-09-13T09:27:56.377 INFO:teuthology.orchestra.run.smithi063.stdout:5636 writesame 0xa360070 thru 0xa36f61a data_size 0x1(0xf5ab bytes) 2024-09-13T09:27:56.377 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:56.378 INFO:teuthology.orchestra.run.smithi063.stdout:5638 writesame 0x7fbbbc2 thru 0x7fca49c data_size 0x1(0xe8db bytes) 2024-09-13T09:27:56.417 INFO:teuthology.orchestra.run.smithi063.stdout:5640 read 0x3756b94 thru 0x375d72b (0x6b98 bytes) 2024-09-13T09:27:56.417 INFO:teuthology.orchestra.run.smithi063.stdout:5641 punch from 0x525be00 to 0x5268b12, (0xcd12 bytes) 2024-09-13T09:27:56.418 INFO:teuthology.orchestra.run.smithi063.stdout:5642 punch from 0x21c8939 to 0x21d14ad, (0x8b74 bytes) 2024-09-13T09:27:56.418 INFO:teuthology.orchestra.run.smithi063.stdout:5643 punch from 0x285c1e7 to 0x285ed08, (0x2b21 bytes) 2024-09-13T09:27:56.418 INFO:teuthology.orchestra.run.smithi063.stdout:5644 punch from 0x9022a4d to 0x9032242, (0xf7f5 bytes) 2024-09-13T09:27:56.418 INFO:teuthology.orchestra.run.smithi063.stdout:5645 read 0x26ffa7e thru 0x270dfcb (0xe54e bytes) 2024-09-13T09:27:56.418 INFO:teuthology.orchestra.run.smithi063.stdout:5646 clone 41 order 24 su 2097152 sc 6 2024-09-13T09:27:56.423 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:56.423 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:56.430 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:56.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:27:56.444 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:56.440+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:56.448 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:56.466 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:56.486 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:56.505 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:27:56.507 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:27:56.516 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:56.520 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:27:56.538 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:56.539 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:56.539 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:56.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:27:56.560 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:56.560 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:56.556+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:56.590 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:56.593 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:56.626 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone40 from 0xa36f61b (overlap 0xc98b2) to 0x509e0 2024-09-13T09:27:56.645 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:56.642+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:56.659 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:56.660 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:56.660 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:56.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:27:56.746 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:56.742+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:56.774 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:297 3.10 3:0ef6f98a:::rbd_data.111867511a81.000000000000005b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 117.213357841, 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.047905149000000001, 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 297 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559130+0000", 2024-09-13T09:27:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559128+0000", 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559145+0000", 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7059999999999999e-05 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559178+0000", 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3318999999999997e-05 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559194+0000", 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5336999999999999e-05 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559284+0000", 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0302000000000004e-05 2024-09-13T09:27:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.559518+0000", 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000234412 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 5", 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.573765+0000", 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01424693 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606776+0000", 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033010932 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606876+0000", 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9840000000000006e-05 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.606972+0000", 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5600999999999995e-05 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:59.607035+0000", 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3924999999999995e-05 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:81133 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1333248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.403134578999996, 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041856440000000002, 2024-09-13T09:27:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 81133 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369354+0000", 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369353+0000", 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369357+0000", 2024-09-13T09:27:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.591e-06 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369365+0000", 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3739999999999998e-06 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.369371+0000", 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1410000000000002e-06 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401839+0000", 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032467408000000003 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.401921+0000", 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1955999999999999e-05 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.411210+0000", 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092894880000000003 2024-09-13T09:27:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84334 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 797696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.144391708000001, 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046676820000000001, 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84334 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628096+0000", 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628099+0000", 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3580000000000001e-06 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628104+0000", 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3510000000000001e-06 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.628108+0000", 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664582+0000", 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036473301999999999 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.664644+0000", 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2632999999999995e-05 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.674773+0000", 2024-09-13T09:27:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010128877999999999 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84353 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 935936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.142189369999997, 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041401856000000001, 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84353 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630299+0000", 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630298+0000", 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630303+0000", 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2089999999999996e-06 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630312+0000", 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8089999999999999e-06 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630325+0000", 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.347e-05 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670284+0000", 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.039958470000000003 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.670351+0000", 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6697000000000002e-05 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671701+0000", 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013501469999999999 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84355 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 951296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.141792769000006, 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041328532000000001, 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84355 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630695+0000", 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630694+0000", 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630700+0000", 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7270000000000004e-06 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630710+0000", 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0149000000000001e-05 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.630718+0000", 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2860000000000002e-06 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671725+0000", 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041007738000000002 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.671792+0000", 2024-09-13T09:27:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672024+0000", 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023145199999999999 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84361 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 997376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.140917384999995, 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041387737000000001, 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84361 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631571+0000", 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631570+0000", 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631574+0000", 2024-09-13T09:27:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631582+0000", 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631587+0000", 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672668+0000", 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041080969000000002 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672733+0000", 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5049999999999996e-05 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672959+0000", 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022551900000000001 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84362 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1012736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.140896401999996, 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041691647999999998, 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84362 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631592+0000", 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631591+0000", 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631594+0000", 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4150000000000002e-06 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631600+0000", 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7810000000000002e-06 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631612+0000", 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2565000000000001e-05 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.672981+0000", 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041368762000000003 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673053+0000", 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2041999999999999e-05 2024-09-13T09:27:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673283+0000", 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023044499999999999 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84364 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1028096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.140845889999994, 2024-09-13T09:27:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041960277999999997, 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84364 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631642+0000", 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631645+0000", 2024-09-13T09:27:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631652+0000", 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7619999999999998e-06 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631658+0000", 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4820000000000002e-06 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673306+0000", 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041648313999999999 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673374+0000", 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8251999999999996e-05 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673603+0000", 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022849 2024-09-13T09:27:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:84366 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1043456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.140816031, 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042250386000000001, 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 84366 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631672+0000", 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631675+0000", 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631682+0000", 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6490000000000002e-06 2024-09-13T09:27:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.631687+0000", 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9660000000000004e-06 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673626+0000", 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041939305000000003 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673694+0000", 2024-09-13T09:27:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7928000000000004e-05 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.673923+0000", 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022875799999999999 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85369 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1033216~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090476348999999, 2024-09-13T09:27:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043572778, 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85369 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682012+0000", 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682011+0000", 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682014+0000", 2024-09-13T09:27:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.446e-06 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682020+0000", 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6060000000000002e-06 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682024+0000", 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5750000000000002e-06 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.715951+0000", 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033926764999999998 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.716004+0000", 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3084999999999997e-05 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725585+0000", 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095806650000000004 2024-09-13T09:27:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85371 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 847872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090417771000006, 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042446913000000003, 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85371 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682070+0000", 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682073+0000", 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:27:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682080+0000", 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9210000000000003e-06 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682085+0000", 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.446e-06 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724323+0000", 2024-09-13T09:27:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042237779000000003 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724398+0000", 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4875000000000004e-05 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724517+0000", 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119171 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85374 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 863232~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090395135999998, 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042622749000000001, 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85374 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682093+0000", 2024-09-13T09:27:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682095+0000", 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682101+0000", 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8370000000000002e-06 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682106+0000", 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7119999999999999e-06 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724540+0000", 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042434516999999998 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724607+0000", 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7095999999999996e-05 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724716+0000", 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108483 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85376 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 878592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090378173999994, 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042803169000000002, 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85376 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682110+0000", 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682112+0000", 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682118+0000", 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682122+0000", 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3710000000000002e-06 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724737+0000", 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042615002999999999 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724805+0000", 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7712000000000005e-05 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724913+0000", 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108458 2024-09-13T09:27:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85379 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 893952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.089506443999994, 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042124963000000001, 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85379 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682982+0000", 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682981+0000", 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682986+0000", 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.682994+0000", 2024-09-13T09:27:56.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6769999999999995e-06 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683000+0000", 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2489999999999998e-06 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724934+0000", 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041933879 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.724999+0000", 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5302000000000006e-05 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725107+0000", 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107493 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85383 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 909312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.089473810000001, 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042301749, 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85383 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683014+0000", 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683017+0000", 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683023+0000", 2024-09-13T09:27:56.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683028+0000", 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725127+0000", 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042098771 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725191+0000", 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4169000000000002e-05 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725316+0000", 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012480799999999999 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85385 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 924672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.089453410999994, 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042472503000000002, 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.847 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85385 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683035+0000", 2024-09-13T09:27:56.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683034+0000", 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683037+0000", 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6280000000000001e-06 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683043+0000", 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683048+0000", 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3159999999999996e-06 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725337+0000", 2024-09-13T09:27:56.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042288805999999998 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725402+0000", 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5089e-05 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725507+0000", 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105395 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85388 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 940032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.089411725999994, 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042625773999999998, 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85388 2024-09-13T09:27:56.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683076+0000", 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683080+0000", 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5480000000000002e-06 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683086+0000", 2024-09-13T09:27:56.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683091+0000", 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1320000000000002e-06 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725528+0000", 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042437184000000003 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725593+0000", 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725702+0000", 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109261 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85391 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 955392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.089279064999999, 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042689423999999997, 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85391 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.857 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683209+0000", 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683208+0000", 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683213+0000", 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683220+0000", 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.683226+0000", 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9989999999999997e-06 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725726+0000", 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.042500147000000002 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725792+0000", 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5727000000000003e-05 2024-09-13T09:27:56.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.725899+0000", 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010655800000000001 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85403 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1032192~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.086928611999994, 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041404952000000002, 2024-09-13T09:27:56.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85403 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685560+0000", 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685559+0000", 2024-09-13T09:27:56.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685562+0000", 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6730000000000001e-06 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685578+0000", 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6368999999999999e-05 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685588+0000", 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3729999999999999e-06 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726788+0000", 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041200329000000001 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726854+0000", 2024-09-13T09:27:56.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6519999999999993e-05 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726965+0000", 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110055 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85407 3.1e 3:7a84f5dd:::rbd_data.111838d2a860.0000000000000007:17 [read 1047552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.086744616999994, 2024-09-13T09:27:56.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041420525999999999, 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85407 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685744+0000", 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685743+0000", 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:56.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685748+0000", 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0610000000000003e-06 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685757+0000", 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0189999999999995e-06 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.685763+0000", 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.726988+0000", 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.041224754000000002 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727053+0000", 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4535999999999993e-05 2024-09-13T09:27:56.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.727164+0000", 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111414 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:56.871 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:56.872 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-13T09:27:56.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:27:56.874 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:56.866+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:56.973 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:56.968+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:57.028 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:57.028 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:57.028 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:27:57.028 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:57.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:27:57.074 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:57.070+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:57.144 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:27:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:27:57.144 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:27:57.144 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:57.154 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:27:57.197 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:57.194+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:57.213 INFO:teuthology.orchestra.run.smithi063.stdout:5647 write 0x63717a8 thru 0x637aff8 (0x9851 bytes) 2024-09-13T09:27:57.235 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.235 INFO:teuthology.orchestra.run.smithi063.stdout:5649 write 0x8d000e0 thru 0x8d099fc (0x991d bytes) 2024-09-13T09:27:57.265 INFO:teuthology.orchestra.run.smithi063.stdout:5651 write 0xbb17081 thru 0xbb17b56 (0xad6 bytes) 2024-09-13T09:27:57.271 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:27:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:27:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:27:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:27:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 134.305938723, 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:27:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:27:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:27:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 71.192718979000006, 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:27:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 71.192320964000004, 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:27:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:27:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.517214088000003, 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:27:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.517199156000004, 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.51716605, 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:27:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:27:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.517149756999999, 2024-09-13T09:27:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:27:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:27:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.517133642000005, 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:27:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.517102601000005, 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:27:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:27:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.517070978000007, 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:27:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.504122917000004, 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:27:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.504105860999999, 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:27:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.504089692999997, 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:27:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.503876031000004, 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:27:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.503856720000002, 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:27:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.503840327000006, 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:27:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.503824664999996, 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:27:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:27:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:27:57.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.503798310999997, 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:57.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.975285584000002, 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:27:57.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:27:57.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:27:57.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:27:57.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.863371469000001, 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:27:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:27:57.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:27:57.369 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:27:57.371 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5652 read 0x6f030e4 thru 0x6f11143 (0xe060 bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5653 read 0xb28035c thru 0xb281767 (0x140c bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5654 write 0x27ab6fe thru 0x27b8004 (0xc907 bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5655 punch from 0x4ef4a6f to 0x4ef503b, (0x5cc bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5656 write 0x3dbc2c3 thru 0x3dc5f52 (0x9c90 bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5658 writesame 0xba30e66 thru 0xba36a91 data_size 0x22(0x5c2c bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5659 read 0x14fb3fd thru 0x1501f6f (0x6b73 bytes) 2024-09-13T09:27:57.373 INFO:teuthology.orchestra.run.smithi063.stdout:5660 read 0x908c326 thru 0x909560d (0x92e8 bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5662 write 0x38880c3 thru 0x38955d1 (0xd50f bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5665 punch from 0x6926dc8 to 0x692e61c, (0x7854 bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5666 read 0x15d1860 thru 0x15de4da (0xcc7b bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5667 write 0x4a3f224 thru 0x4a42932 (0x370f bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5668 punch from 0x5aa497b to 0x5aab1b9, (0x683e bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5669 read 0x491e21e thru 0x492e16e (0xff51 bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5671 read 0x567899b thru 0x56790c0 (0x726 bytes) 2024-09-13T09:27:57.374 INFO:teuthology.orchestra.run.smithi063.stdout:5672 read 0x2bfa9a5 thru 0x2c003a3 (0x59ff bytes) 2024-09-13T09:27:57.374 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:57.364+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:57.375 INFO:teuthology.orchestra.run.smithi063.stdout:5673 trunc from 0xbb17b57 to 0xbf22898 2024-09-13T09:27:57.396 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.396 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.396 INFO:teuthology.orchestra.run.smithi063.stdout:5677 read 0x6cee968 thru 0x6cf53eb (0x6a84 bytes) 2024-09-13T09:27:57.397 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.397 INFO:teuthology.orchestra.run.smithi063.stdout:5681 punch from 0x6479c09 to 0x6479e7f, (0x276 bytes) 2024-09-13T09:27:57.397 INFO:teuthology.orchestra.run.smithi063.stdout:5683 punch from 0x5f1a06a to 0x5f1a2cb, (0x261 bytes) 2024-09-13T09:27:57.398 INFO:teuthology.orchestra.run.smithi063.stdout:5685 read 0x5745396 thru 0x5751f9c (0xcc07 bytes) 2024-09-13T09:27:57.398 INFO:teuthology.orchestra.run.smithi063.stdout:5686 read 0xc126ca thru 0xc14eef (0x2826 bytes) 2024-09-13T09:27:57.398 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.414 INFO:teuthology.orchestra.run.smithi063.stdout:5689 writesame 0xcdd610c thru 0xcdd91c2 data_size 0x1(0x30b7 bytes) 2024-09-13T09:27:57.414 INFO:teuthology.orchestra.run.smithi063.stdout:5690 punch from 0x592f9f9 to 0x5933e28, (0x442f bytes) 2024-09-13T09:27:57.414 INFO:teuthology.orchestra.run.smithi063.stdout:5692 read 0x9bcf3f0 thru 0x9bd3e7d (0x4a8e bytes) 2024-09-13T09:27:57.416 INFO:teuthology.orchestra.run.smithi063.stdout:5693 writesame 0x78116c5 thru 0x7814ab8 data_size 0x19(0x33f4 bytes) 2024-09-13T09:27:57.421 INFO:teuthology.orchestra.run.smithi063.stdout:5694 punch from 0xcc700c2 to 0xcc712df, (0x121d bytes) 2024-09-13T09:27:57.422 INFO:teuthology.orchestra.run.smithi063.stdout:5695 read 0x8d6d42f thru 0x8d773c2 (0x9f94 bytes) 2024-09-13T09:27:57.428 INFO:teuthology.orchestra.run.smithi063.stdout:5696 trunc from 0xcdd91c3 to 0xa1f0efc 2024-09-13T09:27:57.451 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.451 INFO:teuthology.orchestra.run.smithi063.stdout:5698 read 0x2a23804 thru 0x2a2ca7c (0x9279 bytes) 2024-09-13T09:27:57.452 INFO:teuthology.orchestra.run.smithi063.stdout:5700 punch from 0x478e37e to 0x479a5dd, (0xc25f bytes) 2024-09-13T09:27:57.452 INFO:teuthology.orchestra.run.smithi063.stdout:5701 write 0x5175e83 thru 0x5184749 (0xe8c7 bytes) 2024-09-13T09:27:57.456 INFO:teuthology.orchestra.run.smithi063.stdout:5702 punch from 0xfade1 to 0x102ef0, (0x810f bytes) 2024-09-13T09:27:57.457 INFO:teuthology.orchestra.run.smithi063.stdout:5703 writesame 0x367229c thru 0x36723c6 data_size 0x1(0x12b bytes) 2024-09-13T09:27:57.460 INFO:teuthology.orchestra.run.smithi063.stdout:5704 write 0x34e242f thru 0x34e7197 (0x4d69 bytes) 2024-09-13T09:27:57.471 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:57.466+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:57.481 INFO:teuthology.orchestra.run.smithi063.stdout:5705 trunc from 0xa1f0efc to 0xb123a26 2024-09-13T09:27:57.504 INFO:teuthology.orchestra.run.smithi063.stdout:5708 write 0xb7c0e3b thru 0xb7c1597 (0x75d bytes) 2024-09-13T09:27:57.507 INFO:teuthology.orchestra.run.smithi063.stdout:5709 write 0xec4329 thru 0xec4cef (0x9c7 bytes) 2024-09-13T09:27:57.511 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.511 INFO:teuthology.orchestra.run.smithi063.stdout:5711 read 0x74283f6 thru 0x742edd1 (0x69dc bytes) 2024-09-13T09:27:57.518 INFO:teuthology.orchestra.run.smithi063.stdout:5712 read 0x71dc3f thru 0x728709 (0xaacb bytes) 2024-09-13T09:27:57.518 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:27:57.519 INFO:teuthology.orchestra.run.smithi063.stdout:5713 punch from 0x48cf6b1 to 0x48da73a, (0xb089 bytes) 2024-09-13T09:27:57.519 INFO:teuthology.orchestra.run.smithi063.stdout:5714 punch from 0x3cdc48a to 0x3ce6848, (0xa3be bytes) 2024-09-13T09:27:57.519 INFO:teuthology.orchestra.run.smithi063.stdout:5715 writesame 0x1b20bc6 thru 0x1b2d0d1 data_size 0x2(0xc50c bytes) 2024-09-13T09:27:57.522 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:57.522 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:57.522 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:57.522 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:57.531 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:27:57.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:27:57.536 INFO:teuthology.orchestra.run.smithi063.stdout:5716 write 0x7e5f7b5 thru 0x7e692b5 (0x9b01 bytes) 2024-09-13T09:27:57.543 INFO:teuthology.orchestra.run.smithi063.stdout:5717 write 0x3de259 thru 0x3dec6d (0xa15 bytes) 2024-09-13T09:27:57.556 INFO:teuthology.orchestra.run.smithi063.stdout:5718 writesame 0xb7c0496 thru 0xb7c4850 data_size 0x1(0x43bb bytes) 2024-09-13T09:27:57.572 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:57.566+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:57.573 INFO:teuthology.orchestra.run.smithi063.stdout:5719 punch from 0xa0439aa to 0xa045782, (0x1dd8 bytes) 2024-09-13T09:27:57.582 INFO:teuthology.orchestra.run.smithi063.stdout:5720 trunc from 0xb7c4851 to 0x47dbac4 2024-09-13T09:27:57.637 INFO:teuthology.orchestra.run.smithi063.stdout:5721 write 0x7806dbe thru 0x7810af7 (0x9d3a bytes) 2024-09-13T09:27:57.653 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:57.653 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:57.653 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:57.653 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:57.653 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:57.665 INFO:teuthology.orchestra.run.smithi063.stdout:5723 write 0xeafb2d6 thru 0xeaffde9 (0x4b14 bytes) 2024-09-13T09:27:57.667 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:27:57.673 INFO:teuthology.orchestra.run.smithi063.stdout:5724 write 0xba9ad4c thru 0xbaa2ada (0x7d8f bytes) 2024-09-13T09:27:57.673 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:57.668+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:57.678 INFO:teuthology.orchestra.run.smithi063.stdout:5725 read 0x11d5687 thru 0x11d8f9f (0x3919 bytes) 2024-09-13T09:27:57.679 INFO:teuthology.orchestra.run.smithi063.stdout:5727 read 0xd3676dd thru 0xd36954e (0x1e72 bytes) 2024-09-13T09:27:57.679 INFO:teuthology.orchestra.run.smithi063.stdout:5728 writesame 0x8b2ae94 thru 0x8b33e51 data_size 0x3(0x8fbe bytes) 2024-09-13T09:27:57.693 INFO:teuthology.orchestra.run.smithi063.stdout:5730 writesame 0x24fe90e thru 0x2504a5f data_size 0x1(0x6152 bytes) 2024-09-13T09:27:57.707 INFO:teuthology.orchestra.run.smithi063.stdout:5731 write 0x56b74bc thru 0x56c6247 (0xed8c bytes) 2024-09-13T09:27:57.710 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:57.710 INFO:teuthology.orchestra.run.smithi063.stdout:5733 read 0xa3e2b01 thru 0xa3e9765 (0x6c65 bytes) 2024-09-13T09:27:57.711 INFO:teuthology.orchestra.run.smithi063.stdout:5734 read 0x441b485 thru 0x4425f7b (0xaaf7 bytes) 2024-09-13T09:27:57.712 INFO:teuthology.orchestra.run.smithi063.stdout:5735 clone 42 order 19 su 8192 sc 6 2024-09-13T09:27:57.773 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:57.768+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:57.777 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 53.035494614000001, 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:57.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:27:57.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.784 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141158 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.229092879, 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039207350000000002, 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141158 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544057+0000", 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8500000000000001e-06 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544062+0000", 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544065+0000", 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4570000000000001e-06 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583152+0000", 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039086837999999999 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583194+0000", 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2286999999999997e-05 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583263+0000", 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8569e-05 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141159 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.229079836, 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039321855000000003, 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141159 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.790 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544070+0000", 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7719999999999999e-06 2024-09-13T09:27:57.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544074+0000", 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4209999999999997e-06 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544079+0000", 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3839999999999999e-06 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583292+0000", 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039213523 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583337+0000", 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5243e-05 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583390+0000", 2024-09-13T09:27:57.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2769999999999998e-05 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141160 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 120832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.229065824999999, 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039426997999999998, 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.795 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141160 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544085+0000", 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.605e-06 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544089+0000", 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9609999999999996e-06 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544093+0000", 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:27:57.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583414+0000", 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039320689999999998 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583455+0000", 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1267000000000003e-05 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583509+0000", 2024-09-13T09:27:57.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4051999999999997e-05 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141161 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 137216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.229051096999999, 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039545311, 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141161 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544099+0000", 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.497e-06 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544105+0000", 2024-09-13T09:27:57.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6080000000000003e-06 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544109+0000", 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2440000000000004e-06 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583533+0000", 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039424167000000003 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583584+0000", 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0470000000000003e-05 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583642+0000", 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141162 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 161792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.804 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.229034839000001, 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039683041000000002, 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141162 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:27:57.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:27:57.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544116+0000", 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.464e-06 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544121+0000", 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.288e-06 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544125+0000", 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1389999999999997e-06 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583663+0000", 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039537566000000003 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583747+0000", 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4695000000000001e-05 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583796+0000", 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9214e-05 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141163 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.229019767, 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039773226000000002, 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141163 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544131+0000", 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.599e-06 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544152+0000", 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1195e-05 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544157+0000", 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2229999999999998e-06 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583819+0000", 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039662206999999998 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583858+0000", 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8745999999999998e-05 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583902+0000", 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3553000000000001e-05 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.813 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141164 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228985933000001, 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039840280999999998, 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141164 2024-09-13T09:27:57.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544165+0000", 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544170+0000", 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6330000000000001e-06 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544174+0000", 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583924+0000", 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039749128000000002 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583962+0000", 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8461000000000003e-05 2024-09-13T09:27:57.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584003+0000", 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0611000000000001e-05 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.818 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141165 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228970367000001, 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039975881999999997, 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141165 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544179+0000", 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.891e-06 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544184+0000", 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:27:57.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544188+0000", 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8879999999999999e-06 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584060+0000", 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039871697999999997 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584106+0000", 2024-09-13T09:27:57.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6199999999999998e-05 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584154+0000", 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7942000000000003e-05 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141166 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228956959, 2024-09-13T09:27:57.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042489992999999997, 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141166 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544193+0000", 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9410000000000001e-06 2024-09-13T09:27:57.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544198+0000", 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.04e-06 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544202+0000", 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0550000000000001e-06 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584193+0000", 2024-09-13T09:27:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039990942000000002 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586600+0000", 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024071449999999999 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586681+0000", 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1153e-05 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141167 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 276480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228942748, 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042703159999999997, 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141167 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544207+0000", 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.956e-06 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544212+0000", 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544216+0000", 2024-09-13T09:27:57.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1819999999999999e-06 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586717+0000", 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042501039999999997 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586828+0000", 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111097 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586909+0000", 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0378999999999997e-05 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141168 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 317440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544220+0000", 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228928558, 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042857105999999999, 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.832 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141168 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544220+0000", 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544220+0000", 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544219+0000", 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544221+0000", 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8190000000000001e-06 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544226+0000", 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:27:57.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544236+0000", 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4520000000000003e-06 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586931+0000", 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042695205 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587002+0000", 2024-09-13T09:27:57.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1062000000000005e-05 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587077+0000", 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4939999999999997e-05 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141169 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 333824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228908939, 2024-09-13T09:27:57.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043015721999999999, 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141169 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544245+0000", 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544251+0000", 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4739999999999996e-06 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544255+0000", 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0829999999999997e-06 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587125+0000", 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042870220000000001 2024-09-13T09:27:57.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587193+0000", 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7898999999999995e-05 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587255+0000", 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1977999999999995e-05 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141170 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228889430000001, 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043172973000000003, 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141170 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544260+0000", 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8330000000000001e-06 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544265+0000", 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0819999999999998e-06 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544270+0000", 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1439999999999996e-06 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587302+0000", 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043032778000000001 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587371+0000", 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8363999999999998e-05 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587432+0000", 2024-09-13T09:27:57.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1067000000000003e-05 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141171 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 374784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228875023000001, 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043357668000000002, 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.845 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141171 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544275+0000", 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.962e-06 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544280+0000", 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544284+0000", 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1450000000000001e-06 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587466+0000", 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043182470000000001 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587547+0000", 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0981000000000003e-05 2024-09-13T09:27:57.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587631+0000", 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3473000000000005e-05 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141172 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 415744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228860911, 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043467455000000002, 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.849 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141172 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544289+0000", 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.88e-06 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544294+0000", 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544298+0000", 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1080000000000003e-06 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587659+0000", 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043360797 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587711+0000", 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2472999999999997e-05 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587755+0000", 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3328000000000003e-05 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.853 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141173 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 432128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544302+0000", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228846491000001, 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043653346000000003, 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141173 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544302+0000", 2024-09-13T09:27:57.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544302+0000", 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544301+0000", 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544303+0000", 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.978e-06 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544322+0000", 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8453000000000001e-05 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544327+0000", 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.707e-06 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587790+0000", 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043463357000000001 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587871+0000", 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0987000000000002e-05 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587955+0000", 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.857 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141174 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228817243, 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043818442999999999, 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141174 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.858 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:27:57.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544333+0000", 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.571e-06 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544339+0000", 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5049999999999998e-06 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544343+0000", 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588002+0000", 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043659011999999997 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588091+0000", 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9777000000000005e-05 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588149+0000", 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8115999999999997e-05 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141175 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228802385, 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.044033668999999998, 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141175 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544348+0000", 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6850000000000001e-06 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544353+0000", 2024-09-13T09:27:57.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8330000000000001e-06 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544357+0000", 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9040000000000002e-06 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588199+0000", 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043841822000000003 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588298+0000", 2024-09-13T09:27:57.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8825000000000003e-05 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588380+0000", 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1905999999999998e-05 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141176 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.228787807, 2024-09-13T09:27:57.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.044145693999999999, 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141176 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544363+0000", 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4210000000000002e-06 2024-09-13T09:27:57.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544367+0000", 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8409999999999999e-06 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544372+0000", 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.233e-06 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588414+0000", 2024-09-13T09:27:57.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.044042247999999999 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588454+0000", 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9696000000000001e-05 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588506+0000", 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2515000000000001e-05 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:57.871 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:57.871 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:57.872 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:27:57.916 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:57.912+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:57.975 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:57.970+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:58.028 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:58.028 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:58.028 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:58.028 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:58.038 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:27:58.077 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:58.074+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:58.149 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:58.149 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:58.149 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:58.149 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:58.149 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:58.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:27:58.177 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:58.172+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:58.269 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:58.269 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:27:58.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.702834350000003, 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.270 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:27:58.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.995265067999995, 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.275 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:27:58.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:27:58.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:27:58.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 44.115175970999999, 2024-09-13T09:27:58.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:27:58.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130769 3.1 3:84b2050c:::rbd_data.1118de6ae5af.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 29.229822222999999, 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024401849, 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130769 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035103+0000", 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035114+0000", 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1613e-05 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035136+0000", 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1472e-05 2024-09-13T09:27:58.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035145+0000", 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4860000000000003e-06 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035204+0000", 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9027000000000003e-05 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035363+0000", 2024-09-13T09:27:58.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015868600000000001 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.040402+0000", 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050390000000000001 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059321+0000", 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018919062 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059340+0000", 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9001999999999999e-05 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059432+0000", 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1535999999999999e-05 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059507+0000", 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5220000000000001e-05 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139806 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 309248~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.879020735999999, 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022069070999999999, 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139806 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.292 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385909+0000", 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.543e-06 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385914+0000", 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.789e-06 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385919+0000", 2024-09-13T09:27:58.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.899e-06 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.407837+0000", 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02191802 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.407916+0000", 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8829e-05 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.407976+0000", 2024-09-13T09:27:58.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9453999999999997e-05 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139807 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.879003726000001, 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022177722, 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139807 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385926+0000", 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.424e-06 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385933+0000", 2024-09-13T09:27:58.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7340000000000008e-06 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385938+0000", 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408007+0000", 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022068910000000001 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408062+0000", 2024-09-13T09:27:58.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5324000000000001e-05 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408101+0000", 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8840000000000001e-05 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139808 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385943+0000", 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878984586, 2024-09-13T09:27:58.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022281553999999999, 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139808 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385943+0000", 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385943+0000", 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385942+0000", 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385945+0000", 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8150000000000002e-06 2024-09-13T09:27:58.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385951+0000", 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0170000000000002e-06 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385956+0000", 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7210000000000002e-06 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408116+0000", 2024-09-13T09:27:58.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022160119999999998 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408177+0000", 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0878000000000002e-05 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408224+0000", 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7355999999999999e-05 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139809 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 391168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878967124000001, 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022375121000000001, 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139809 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385962+0000", 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.272e-06 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385975+0000", 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2604e-05 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385986+0000", 2024-09-13T09:27:58.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1049e-05 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408258+0000", 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022272734999999998 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408299+0000", 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0599999999999998e-05 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408335+0000", 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6186000000000002e-05 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139810 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 407552~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386011+0000", 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878916276, 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022448594999999998, 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139810 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386011+0000", 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386011+0000", 2024-09-13T09:27:58.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386010+0000", 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386014+0000", 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1769999999999998e-06 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386026+0000", 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2289999999999999e-05 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386036+0000", 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0244000000000001e-05 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408349+0000", 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022313146999999998 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408418+0000", 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9099999999999999e-05 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408460+0000", 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1156999999999999e-05 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139811 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 448512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.313 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878885029999999, 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022522297, 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139811 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:27:58.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386045+0000", 2024-09-13T09:27:58.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8420000000000001e-06 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386051+0000", 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386056+0000", 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8100000000000003e-06 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408474+0000", 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022417245999999998 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408528+0000", 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4861999999999998e-05 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408564+0000", 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6022999999999997e-05 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139812 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 464896~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878865961000001, 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022630847999999999, 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139812 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.318 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:27:58.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386064+0000", 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6919999999999998e-06 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386070+0000", 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2040000000000002e-06 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386075+0000", 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9640000000000002e-06 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408581+0000", 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02250628 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408653+0000", 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2151999999999996e-05 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408692+0000", 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8923e-05 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139813 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 489472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878848016999999, 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022729668000000001, 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139813 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:27:58.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386081+0000", 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386087+0000", 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386092+0000", 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408710+0000", 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022617624999999999 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408762+0000", 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2592000000000001e-05 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408809+0000", 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6519e-05 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.326 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139814 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 505856~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878830776999999, 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022830249, 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139814 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386105+0000", 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7819999999999999e-06 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386112+0000", 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6880000000000002e-06 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386117+0000", 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408823+0000", 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022706271 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408886+0000", 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2645999999999996e-05 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408927+0000", 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0911999999999997e-05 2024-09-13T09:27:58.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139815 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386122+0000", 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878805138000001, 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029220356999999999, 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139815 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386122+0000", 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386122+0000", 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386121+0000", 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386125+0000", 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7519999999999998e-06 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386131+0000", 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386136+0000", 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0379999999999999e-06 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408941+0000", 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022804334999999998 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409017+0000", 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6668999999999994e-05 2024-09-13T09:27:58.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409287+0000", 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026946099999999997 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413128+0000", 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003840979 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415236+0000", 2024-09-13T09:27:58.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021087279999999998 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415293+0000", 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6660999999999998e-05 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415342+0000", 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9279e-05 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139816 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878786052000001, 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02928824, 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139816 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386144+0000", 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6119999999999999e-06 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386150+0000", 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0719999999999996e-06 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386155+0000", 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9779999999999996e-06 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409919+0000", 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.023763375999999999 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409981+0000", 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2708999999999999e-05 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.410214+0000", 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023281800000000001 2024-09-13T09:27:58.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413143+0000", 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029293129999999998 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415351+0000", 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022077099999999999 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415391+0000", 2024-09-13T09:27:58.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9702e-05 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415429+0000", 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8494000000000001e-05 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139817 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386160+0000", 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878767643, 2024-09-13T09:27:58.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029572294999999998, 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139817 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386160+0000", 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386160+0000", 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386159+0000", 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386162+0000", 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9869999999999999e-06 2024-09-13T09:27:58.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386168+0000", 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4110000000000004e-06 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386172+0000", 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.809e-06 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.410784+0000", 2024-09-13T09:27:58.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024611091000000002 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.410848+0000", 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4542999999999995e-05 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.411102+0000", 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025437799999999998 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413162+0000", 2024-09-13T09:27:58.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020593289999999999 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415668+0000", 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025066630000000001 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415698+0000", 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9192000000000001e-05 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415732+0000", 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4239999999999997e-05 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139818 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878750303, 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030114272000000001, 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.348 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139818 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386180+0000", 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1259999999999998e-06 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386185+0000", 2024-09-13T09:27:58.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5790000000000003e-06 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386190+0000", 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1499999999999998e-06 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.411771+0000", 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025581115000000001 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.411843+0000", 2024-09-13T09:27:58.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1823999999999995e-05 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412077+0000", 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023346400000000001 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415093+0000", 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003016482 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416209+0000", 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011161369999999999 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416256+0000", 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6456999999999999e-05 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416291+0000", 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5330999999999998e-05 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139819 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.878731994000001, 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030190423000000001, 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139819 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.354 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:27:58.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386205+0000", 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0128e-05 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386211+0000", 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386216+0000", 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.899e-06 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412527+0000", 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.026310579000000001 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412573+0000", 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6637000000000001e-05 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412749+0000", 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017502199999999999 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415137+0000", 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023887359999999998 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416302+0000", 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001165234 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416347+0000", 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4428000000000002e-05 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416386+0000", 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8733000000000003e-05 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139820 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.393356+0000", 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.871570976999999, 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.023103754000000001, 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139820 2024-09-13T09:27:58.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393356+0000", 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393356+0000", 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393354+0000", 2024-09-13T09:27:58.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393365+0000", 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0968000000000001e-05 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393379+0000", 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393393+0000", 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3900000000000001e-05 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413174+0000", 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019781499000000001 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413218+0000", 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3829000000000003e-05 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413500+0000", 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028210300000000002 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415552+0000", 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020520780000000001 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416396+0000", 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00084346599999999996 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416430+0000", 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4542999999999998e-05 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416460+0000", 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9802e-05 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139821 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:27:58.365 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.394603+0000", 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.870323853, 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.021986615000000001, 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139821 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394603+0000", 2024-09-13T09:27:58.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394603+0000", 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394602+0000", 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394608+0000", 2024-09-13T09:27:58.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6230000000000002e-06 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394622+0000", 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3963e-05 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394630+0000", 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2779999999999993e-06 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:27:58.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.414023+0000", 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019392857999999999 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.414111+0000", 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7719000000000005e-05 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.414481+0000", 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036972300000000002 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416165+0000", 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016848169999999999 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416473+0000", 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030729999999999999 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416546+0000", 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3529999999999996e-05 2024-09-13T09:27:58.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416590+0000", 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3683000000000002e-05 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:58.371 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:58.373 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:27:58.373 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:58.368+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:58.473 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:58.470+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:58.522 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:27:58.534 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:27:58.546 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:58.546 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:58.546 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:27:58.546 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:58.550 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:27:58.551 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:27:58.558 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:27:58.574 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:58.570+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:58.605 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:27:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone41 from 0xeaffdea (overlap 0x47dbac4) to 0x0 2024-09-13T09:27:58.675 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:58.668+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:58.726 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:58.726 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:27:58.726 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:27:58.726 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:27:58.727 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:58.739 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:27:58.763 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:27:58.774 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:58.768+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:58.775 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-13T09:27:58.784 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:27:58.799 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:58.799 INFO:teuthology.orchestra.run.smithi063.stdout:5738 write 0x9c1e87a thru 0x9c2bc71 (0xd3f8 bytes) 2024-09-13T09:27:58.806 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:27:58.829 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:27:58.863 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:58.863 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:58.863 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:27:58.876 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:58.868+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:58.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:27:58.878 INFO:teuthology.orchestra.run.smithi063.stdout:5739 write 0x95db5cd thru 0x95de843 (0x3277 bytes) 2024-09-13T09:27:58.887 INFO:teuthology.orchestra.run.smithi063.stdout:5741 writesame 0x18b81d thru 0x19b392 data_size 0x3(0xfb76 bytes) 2024-09-13T09:27:58.900 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:27:58.923 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:27:58.944 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:27:58.946 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:27:58.946 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:27:58.946 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:27:58.946 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:27:58.946 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:27:58.947 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:27:58.947 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:27:58.947 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:27:58.983 INFO:teuthology.orchestra.run.smithi063.stdout:5742 trunc from 0xeaffdea to 0xd42efc 2024-09-13T09:27:58.983 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:27:58.976+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:27:58.992 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:58.992 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:58.997 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:59.002 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:27:59.003 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:27:59.016 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-13T09:27:58.554084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1535850345,"num_objects":1776,"num_object_clones":342,"num_object_copies":3552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1776,"num_whiteouts":264,"num_read":212971,"num_read_kb":660078,"num_write":27130,"num_write_kb":399473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3084,"num_bytes_recovered":211673521,"num_keys_recovered":658,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20134,"ondisk_log_size":20134,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":470940,"kb_used_data":91332,"kb_used_omap":25,"kb_used_meta":379366,"kb_avail":754503780,"statfs":{"total":773094113280,"available":772611870720,"internally_reserved":0,"allocated":93523968,"data_stored":479863385,"data_compressed":40519005,"data_compressed_allocated":45871104,"data_compressed_original":442182383,"omap_allocated":25946,"internal_metadata":388471462},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,28],"upper_bound":4},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11397597,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":13616,"num_read_kb":12079,"num_write":5171,"num_write_kb":9678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001065"},"pg_stats":[{"pgid":"3.1f","version":"121'430","reported_seq":1728,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.403356+0000","last_change":"2024-09-13T09:26:39.600998+0000","last_active":"2024-09-13T09:27:56.403356+0000","last_peered":"2024-09-13T09:27:56.403356+0000","last_clean":"2024-09-13T09:27:56.403356+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:27:56.403356+0000","last_undegraded":"2024-09-13T09:27:56.403356+0000","last_fullsized":"2024-09-13T09:27:56.403356+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":41177473,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":11,"num_read":2987,"num_read_kb":15929,"num_write":464,"num_write_kb":7559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"121'679","reported_seq":5271,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.486612+0000","last_change":"2024-09-13T09:27:05.744365+0000","last_active":"2024-09-13T09:27:55.486612+0000","last_peered":"2024-09-13T09:27:55.486612+0000","last_clean":"2024-09-13T09:27:55.486612+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:27:55.486612+0000","last_undegraded":"2024-09-13T09:27:55.486612+0000","last_fullsized":"2024-09-13T09:27:55.486612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":34456411,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":4,"num_read":6494,"num_read_kb":13244,"num_write":1002,"num_write_kb":6773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"122'460","reported_seq":4057,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.558900+0000","last_change":"2024-09-13T09:27:39.353427+0000","last_active":"2024-09-13T09:27:56.558900+0000","last_peered":"2024-09-13T09:27:56.558900+0000","last_clean":"2024-09-13T09:27:56.558900+0000","last_became_active":"2024-09-13T09:27:39.352330+0000","last_became_peered":"2024-09-13T09:27:39.352330+0000","last_unstale":"2024-09-13T09:27:56.558900+0000","last_undegraded":"2024-09-13T09:27:56.558900+0000","last_fullsized":"2024-09-13T09:27:56.558900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:56.346789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":57790951,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":9,"num_read":4071,"num_read_kb":20268,"num_write":541,"num_write_kb":11276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":12545300,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"122'506","reported_seq":5119,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.561886+0000","last_change":"2024-09-13T09:27:38.348001+0000","last_active":"2024-09-13T09:27:56.561886+0000","last_peered":"2024-09-13T09:27:56.561886+0000","last_clean":"2024-09-13T09:27:56.561886+0000","last_became_active":"2024-09-13T09:27:38.346735+0000","last_became_peered":"2024-09-13T09:27:38.346735+0000","last_unstale":"2024-09-13T09:27:56.561886+0000","last_undegraded":"2024-09-13T09:27:56.561886+0000","last_fullsized":"2024-09-13T09:27:56.561886+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:23.094925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":67602354,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":8,"num_read":4956,"num_read_kb":17791,"num_write":644,"num_write_kb":13016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":15156139,"num_keys_recovered":138,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"122'480","reported_seq":2899,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501631+0000","last_change":"2024-09-13T09:27:40.167421+0000","last_active":"2024-09-13T09:27:56.501631+0000","last_peered":"2024-09-13T09:27:56.501631+0000","last_clean":"2024-09-13T09:27:56.501631+0000","last_became_active":"2024-09-13T09:27:40.165702+0000","last_became_peered":"2024-09-13T09:27:40.165702+0000","last_unstale":"2024-09-13T09:27:56.501631+0000","last_undegraded":"2024-09-13T09:27:56.501631+0000","last_fullsized":"2024-09-13T09:27:56.501631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:21:08.885741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201516,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":12,"num_read":3733,"num_read_kb":10627,"num_write":552,"num_write_kb":6214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9930624,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"121'550","reported_seq":11532,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.568142+0000","last_change":"2024-09-13T09:27:37.336790+0000","last_active":"2024-09-13T09:27:56.568142+0000","last_peered":"2024-09-13T09:27:56.568142+0000","last_clean":"2024-09-13T09:27:56.568142+0000","last_became_active":"2024-09-13T09:27:37.334754+0000","last_became_peered":"2024-09-13T09:27:37.334754+0000","last_unstale":"2024-09-13T09:27:56.568142+0000","last_undegraded":"2024-09-13T09:27:56.568142+0000","last_fullsized":"2024-09-13T09:27:56.568142+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:54.865010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":46971248,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":6,"num_read":12925,"num_read_kb":21496,"num_write":634,"num_write_kb":6617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":8205452,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"121'507","reported_seq":2623,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.415353+0000","last_change":"2024-09-13T09:27:05.741503+0000","last_active":"2024-09-13T09:27:56.415353+0000","last_peered":"2024-09-13T09:27:56.415353+0000","last_clean":"2024-09-13T09:27:56.415353+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:27:56.415353+0000","last_undegraded":"2024-09-13T09:27:56.415353+0000","last_fullsized":"2024-09-13T09:27:56.415353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":29443717,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":7,"num_read":3351,"num_read_kb":9986,"num_write":535,"num_write_kb":8021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"121'490","reported_seq":1666,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.614985+0000","last_change":"2024-09-13T09:27:03.781188+0000","last_active":"2024-09-13T09:27:55.614985+0000","last_peered":"2024-09-13T09:27:55.614985+0000","last_clean":"2024-09-13T09:27:55.614985+0000","last_became_active":"2024-09-13T09:27:03.711948+0000","last_became_peered":"2024-09-13T09:27:03.711948+0000","last_unstale":"2024-09-13T09:27:55.614985+0000","last_undegraded":"2024-09-13T09:27:55.614985+0000","last_fullsized":"2024-09-13T09:27:55.614985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:42.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":29701076,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":6,"num_read":3304,"num_read_kb":15167,"num_write":512,"num_write_kb":13675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":8502479,"num_keys_recovered":39,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"121'609","reported_seq":1524,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.601987+0000","last_change":"2024-09-13T09:26:42.575210+0000","last_active":"2024-09-13T09:27:55.601987+0000","last_peered":"2024-09-13T09:27:55.601987+0000","last_clean":"2024-09-13T09:27:55.601987+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:27:55.601987+0000","last_undegraded":"2024-09-13T09:27:55.601987+0000","last_fullsized":"2024-09-13T09:27:55.601987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:47.404587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10535024,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":5,"num_read":3079,"num_read_kb":4895,"num_write":723,"num_write_kb":3524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"122'441","reported_seq":5705,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.522068+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:27:56.522068+0000","last_peered":"2024-09-13T09:27:56.522068+0000","last_clean":"2024-09-13T09:27:56.522068+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:27:56.522068+0000","last_undegraded":"2024-09-13T09:27:56.522068+0000","last_fullsized":"2024-09-13T09:27:56.522068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":53405795,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":8,"num_read":5562,"num_read_kb":29365,"num_write":556,"num_write_kb":16204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"121'387","reported_seq":7499,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417871+0000","last_change":"2024-09-13T09:27:05.750137+0000","last_active":"2024-09-13T09:27:56.417871+0000","last_peered":"2024-09-13T09:27:56.417871+0000","last_clean":"2024-09-13T09:27:56.417871+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:27:56.417871+0000","last_undegraded":"2024-09-13T09:27:56.417871+0000","last_fullsized":"2024-09-13T09:27:56.417871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":26932203,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":6,"num_read":7428,"num_read_kb":26689,"num_write":415,"num_write_kb":8419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"121'429","reported_seq":5544,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.087395+0000","last_change":"2024-09-13T09:27:37.337476+0000","last_active":"2024-09-13T09:27:56.087395+0000","last_peered":"2024-09-13T09:27:56.087395+0000","last_clean":"2024-09-13T09:27:56.087395+0000","last_became_active":"2024-09-13T09:27:37.335199+0000","last_became_peered":"2024-09-13T09:27:37.335199+0000","last_unstale":"2024-09-13T09:27:56.087395+0000","last_undegraded":"2024-09-13T09:27:56.087395+0000","last_fullsized":"2024-09-13T09:27:56.087395+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:24:06.331221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":46596591,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":8,"num_read":5711,"num_read_kb":12853,"num_write":460,"num_write_kb":5527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":4959398,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":132,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501625+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:27:56.501625+0000","last_peered":"2024-09-13T09:27:56.501625+0000","last_clean":"2024-09-13T09:27:56.501625+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:27:56.501625+0000","last_undegraded":"2024-09-13T09:27:56.501625+0000","last_fullsized":"2024-09-13T09:27:56.501625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"121'420","reported_seq":1244,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.415720+0000","last_change":"2024-09-13T09:27:05.753951+0000","last_active":"2024-09-13T09:27:56.415720+0000","last_peered":"2024-09-13T09:27:56.415720+0000","last_clean":"2024-09-13T09:27:56.415720+0000","last_became_active":"2024-09-13T09:27:05.753131+0000","last_became_peered":"2024-09-13T09:27:05.753131+0000","last_unstale":"2024-09-13T09:27:56.415720+0000","last_undegraded":"2024-09-13T09:27:56.415720+0000","last_fullsized":"2024-09-13T09:27:56.415720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:21.776084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":69325118,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":8,"num_read":1004,"num_read_kb":6232,"num_write":503,"num_write_kb":8017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501774+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:27:56.501774+0000","last_peered":"2024-09-13T09:27:56.501774+0000","last_clean":"2024-09-13T09:27:56.501774+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:27:56.501774+0000","last_undegraded":"2024-09-13T09:27:56.501774+0000","last_fullsized":"2024-09-13T09:27:56.501774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"121'710","reported_seq":3200,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.395622+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:27:56.395622+0000","last_peered":"2024-09-13T09:27:56.395622+0000","last_clean":"2024-09-13T09:27:56.395622+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:27:56.395622+0000","last_undegraded":"2024-09-13T09:27:56.395622+0000","last_fullsized":"2024-09-13T09:27:56.395622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":35108989,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":6,"num_read":5051,"num_read_kb":18443,"num_write":993,"num_write_kb":21566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.481556+0000","last_change":"2024-09-13T09:27:34.289812+0000","last_active":"2024-09-13T09:27:55.481556+0000","last_peered":"2024-09-13T09:27:55.481556+0000","last_clean":"2024-09-13T09:27:55.481556+0000","last_became_active":"2024-09-13T09:27:34.289466+0000","last_became_peered":"2024-09-13T09:27:34.289466+0000","last_unstale":"2024-09-13T09:27:55.481556+0000","last_undegraded":"2024-09-13T09:27:55.481556+0000","last_fullsized":"2024-09-13T09:27:55.481556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:42.497510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"121'652","reported_seq":5586,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.486868+0000","last_change":"2024-09-13T09:27:05.741382+0000","last_active":"2024-09-13T09:27:55.486868+0000","last_peered":"2024-09-13T09:27:55.486868+0000","last_clean":"2024-09-13T09:27:55.486868+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:27:55.486868+0000","last_undegraded":"2024-09-13T09:27:55.486868+0000","last_fullsized":"2024-09-13T09:27:55.486868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":74457018,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":9,"num_read":7238,"num_read_kb":25922,"num_write":963,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:54.472800+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:27:54.472800+0000","last_peered":"2024-09-13T09:27:54.472800+0000","last_clean":"2024-09-13T09:27:54.472800+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:27:54.472800+0000","last_undegraded":"2024-09-13T09:27:54.472800+0000","last_fullsized":"2024-09-13T09:27:54.472800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"121'689","reported_seq":5426,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.411416+0000","last_change":"2024-09-13T09:26:42.570158+0000","last_active":"2024-09-13T09:27:56.411416+0000","last_peered":"2024-09-13T09:27:56.411416+0000","last_clean":"2024-09-13T09:27:56.411416+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:27:56.411416+0000","last_undegraded":"2024-09-13T09:27:56.411416+0000","last_fullsized":"2024-09-13T09:27:56.411416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53785345,"num_objects":66,"num_object_clones":12,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":10,"num_read":7565,"num_read_kb":24285,"num_write":904,"num_write_kb":8412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"121'1254","reported_seq":2685,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417251+0000","last_change":"2024-09-13T09:27:40.165078+0000","last_active":"2024-09-13T09:27:56.417251+0000","last_peered":"2024-09-13T09:27:56.417251+0000","last_clean":"2024-09-13T09:27:56.417251+0000","last_became_active":"2024-09-13T09:27:40.164156+0000","last_became_peered":"2024-09-13T09:27:40.164156+0000","last_unstale":"2024-09-13T09:27:56.417251+0000","last_undegraded":"2024-09-13T09:27:56.417251+0000","last_fullsized":"2024-09-13T09:27:56.417251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:58.977950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13597679,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":6,"num_read":6916,"num_read_kb":15309,"num_write":1974,"num_write_kb":6812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":6885568,"num_keys_recovered":105,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":263,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490814+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:27:55.490814+0000","last_peered":"2024-09-13T09:27:55.490814+0000","last_clean":"2024-09-13T09:27:55.490814+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:27:55.490814+0000","last_undegraded":"2024-09-13T09:27:55.490814+0000","last_fullsized":"2024-09-13T09:27:55.490814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":119,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:54.482791+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:27:54.482791+0000","last_peered":"2024-09-13T09:27:54.482791+0000","last_clean":"2024-09-13T09:27:54.482791+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:27:54.482791+0000","last_undegraded":"2024-09-13T09:27:54.482791+0000","last_fullsized":"2024-09-13T09:27:54.482791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"121'889","reported_seq":8398,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.080626+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:27:56.080626+0000","last_peered":"2024-09-13T09:27:56.080626+0000","last_clean":"2024-09-13T09:27:56.080626+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:27:56.080626+0000","last_undegraded":"2024-09-13T09:27:56.080626+0000","last_fullsized":"2024-09-13T09:27:56.080626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":64588489,"num_objects":58,"num_object_clones":10,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":9,"num_read":10690,"num_read_kb":23133,"num_write":1323,"num_write_kb":18418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":145,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490844+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:27:55.490844+0000","last_peered":"2024-09-13T09:27:55.490844+0000","last_clean":"2024-09-13T09:27:55.490844+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:27:55.490844+0000","last_undegraded":"2024-09-13T09:27:55.490844+0000","last_fullsized":"2024-09-13T09:27:55.490844+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"121'539","reported_seq":10201,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417876+0000","last_change":"2024-09-13T09:27:38.350122+0000","last_active":"2024-09-13T09:27:56.417876+0000","last_peered":"2024-09-13T09:27:56.417876+0000","last_clean":"2024-09-13T09:27:56.417876+0000","last_became_active":"2024-09-13T09:27:38.348807+0000","last_became_peered":"2024-09-13T09:27:38.348807+0000","last_unstale":"2024-09-13T09:27:56.417876+0000","last_undegraded":"2024-09-13T09:27:56.417876+0000","last_fullsized":"2024-09-13T09:27:56.417876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:08.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":86966958,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":11,"num_read":9772,"num_read_kb":38350,"num_write":673,"num_write_kb":31494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":37856068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":136,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.497662+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:27:55.497662+0000","last_peered":"2024-09-13T09:27:55.497662+0000","last_clean":"2024-09-13T09:27:55.497662+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:27:55.497662+0000","last_undegraded":"2024-09-13T09:27:55.497662+0000","last_fullsized":"2024-09-13T09:27:55.497662+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"121'1861","reported_seq":6509,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.694854+0000","last_change":"2024-09-13T09:27:05.745925+0000","last_active":"2024-09-13T09:27:55.694854+0000","last_peered":"2024-09-13T09:27:55.694854+0000","last_clean":"2024-09-13T09:27:55.694854+0000","last_became_active":"2024-09-13T09:27:05.744803+0000","last_became_peered":"2024-09-13T09:27:05.744803+0000","last_unstale":"2024-09-13T09:27:55.694854+0000","last_undegraded":"2024-09-13T09:27:55.694854+0000","last_fullsized":"2024-09-13T09:27:55.694854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1861,"log_dups_size":0,"ondisk_log_size":1861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:40.185419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":25584455,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":5,"num_read":13735,"num_read_kb":16194,"num_write":3351,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":135,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490772+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:27:55.490772+0000","last_peered":"2024-09-13T09:27:55.490772+0000","last_clean":"2024-09-13T09:27:55.490772+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:27:55.490772+0000","last_undegraded":"2024-09-13T09:27:55.490772+0000","last_fullsized":"2024-09-13T09:27:55.490772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"120'486","reported_seq":1871,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.460423+0000","last_change":"2024-09-13T09:27:05.745269+0000","last_active":"2024-09-13T09:27:55.460423+0000","last_peered":"2024-09-13T09:27:55.460423+0000","last_clean":"2024-09-13T09:27:55.460423+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:27:55.460423+0000","last_undegraded":"2024-09-13T09:27:55.460423+0000","last_fullsized":"2024-09-13T09:27:55.460423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":23606372,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":9,"num_read":2940,"num_read_kb":13676,"num_write":552,"num_write_kb":6739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"121'398","reported_seq":11067,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417645+0000","last_change":"2024-09-13T09:27:39.357803+0000","last_active":"2024-09-13T09:27:56.417645+0000","last_peered":"2024-09-13T09:27:56.417645+0000","last_clean":"2024-09-13T09:27:56.417645+0000","last_became_active":"2024-09-13T09:27:39.355670+0000","last_became_peered":"2024-09-13T09:27:39.355670+0000","last_unstale":"2024-09-13T09:27:56.417645+0000","last_undegraded":"2024-09-13T09:27:56.417645+0000","last_fullsized":"2024-09-13T09:27:56.417645+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:18.749072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":55902485,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":10,"num_read":10451,"num_read_kb":21041,"num_write":430,"num_write_kb":14364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":9133614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"121'382","reported_seq":3629,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.112269+0000","last_change":"2024-09-13T09:27:39.354051+0000","last_active":"2024-09-13T09:27:56.112269+0000","last_peered":"2024-09-13T09:27:56.112269+0000","last_clean":"2024-09-13T09:27:56.112269+0000","last_became_active":"2024-09-13T09:27:39.352709+0000","last_became_peered":"2024-09-13T09:27:39.352709+0000","last_unstale":"2024-09-13T09:27:56.112269+0000","last_undegraded":"2024-09-13T09:27:56.112269+0000","last_fullsized":"2024-09-13T09:27:56.112269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:01.998737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":38529235,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":9,"num_read":3008,"num_read_kb":20333,"num_write":391,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":5132540,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"121'512","reported_seq":4076,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.633966+0000","last_change":"2024-09-13T09:27:05.743332+0000","last_active":"2024-09-13T09:27:55.633966+0000","last_peered":"2024-09-13T09:27:55.633966+0000","last_clean":"2024-09-13T09:27:55.633966+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:27:55.633966+0000","last_undegraded":"2024-09-13T09:27:55.633966+0000","last_fullsized":"2024-09-13T09:27:55.633966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":68568851,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":10,"num_read":3792,"num_read_kb":27296,"num_write":599,"num_write_kb":19857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"121'527","reported_seq":7283,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.418174+0000","last_change":"2024-09-13T09:27:05.746491+0000","last_active":"2024-09-13T09:27:56.418174+0000","last_peered":"2024-09-13T09:27:56.418174+0000","last_clean":"2024-09-13T09:27:56.418174+0000","last_became_active":"2024-09-13T09:27:05.745595+0000","last_became_peered":"2024-09-13T09:27:05.745595+0000","last_unstale":"2024-09-13T09:27:56.418174+0000","last_undegraded":"2024-09-13T09:27:56.418174+0000","last_fullsized":"2024-09-13T09:27:56.418174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:38.773883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":42585420,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":4,"num_read":7699,"num_read_kb":21253,"num_write":745,"num_write_kb":15259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"121'398","reported_seq":14221,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.401322+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:27:56.401322+0000","last_peered":"2024-09-13T09:27:56.401322+0000","last_clean":"2024-09-13T09:27:56.401322+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:27:56.401322+0000","last_undegraded":"2024-09-13T09:27:56.401322+0000","last_fullsized":"2024-09-13T09:27:56.401322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":82298447,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":14,"num_read":13865,"num_read_kb":35794,"num_write":479,"num_write_kb":29504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"121'1075","reported_seq":8014,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417703+0000","last_change":"2024-09-13T09:27:40.166348+0000","last_active":"2024-09-13T09:27:56.417703+0000","last_peered":"2024-09-13T09:27:56.417703+0000","last_clean":"2024-09-13T09:27:56.417703+0000","last_became_active":"2024-09-13T09:27:40.164774+0000","last_became_peered":"2024-09-13T09:27:40.164774+0000","last_unstale":"2024-09-13T09:27:56.417703+0000","last_undegraded":"2024-09-13T09:27:56.417703+0000","last_fullsized":"2024-09-13T09:27:56.417703+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:43.498002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":77899202,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":12,"num_read":10748,"num_read_kb":23563,"num_write":1746,"num_write_kb":18987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":13609565,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"121'411","reported_seq":6266,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.202806+0000","last_change":"2024-09-13T09:27:37.336194+0000","last_active":"2024-09-13T09:27:56.202806+0000","last_peered":"2024-09-13T09:27:56.202806+0000","last_clean":"2024-09-13T09:27:56.202806+0000","last_became_active":"2024-09-13T09:27:37.335370+0000","last_became_peered":"2024-09-13T09:27:37.335370+0000","last_unstale":"2024-09-13T09:27:56.202806+0000","last_undegraded":"2024-09-13T09:27:56.202806+0000","last_fullsized":"2024-09-13T09:27:56.202806+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:20.967570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":59679891,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":6,"num_read":5598,"num_read_kb":29560,"num_write":462,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":13410696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"121'761","reported_seq":3605,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417618+0000","last_change":"2024-09-13T09:26:42.574072+0000","last_active":"2024-09-13T09:27:56.417618+0000","last_peered":"2024-09-13T09:27:56.417618+0000","last_clean":"2024-09-13T09:27:56.417618+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:27:56.417618+0000","last_undegraded":"2024-09-13T09:27:56.417618+0000","last_fullsized":"2024-09-13T09:27:56.417618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":37693572,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":10,"num_read":4846,"num_read_kb":23323,"num_write":1104,"num_write_kb":9498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"121'705","reported_seq":7674,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417713+0000","last_change":"2024-09-13T09:27:41.168314+0000","last_active":"2024-09-13T09:27:56.417713+0000","last_peered":"2024-09-13T09:27:56.417713+0000","last_clean":"2024-09-13T09:27:56.417713+0000","last_became_active":"2024-09-13T09:27:41.165699+0000","last_became_peered":"2024-09-13T09:27:41.165699+0000","last_unstale":"2024-09-13T09:27:56.417713+0000","last_undegraded":"2024-09-13T09:27:56.417713+0000","last_fullsized":"2024-09-13T09:27:56.417713+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:03.888324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":49286710,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":9,"num_read":9644,"num_read_kb":25247,"num_write":885,"num_write_kb":8751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":9211408,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"121'439","reported_seq":5275,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.242228+0000","last_change":"2024-09-13T09:27:17.867086+0000","last_active":"2024-09-13T09:27:56.242228+0000","last_peered":"2024-09-13T09:27:56.242228+0000","last_clean":"2024-09-13T09:27:56.242228+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:27:56.242228+0000","last_undegraded":"2024-09-13T09:27:56.242228+0000","last_fullsized":"2024-09-13T09:27:56.242228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57888450,"num_objects":55,"num_object_clones":7,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":4,"num_read":5013,"num_read_kb":22896,"num_write":583,"num_write_kb":14850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"122'938","reported_seq":7127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.557018+0000","last_change":"2024-09-13T09:27:39.358188+0000","last_active":"2024-09-13T09:27:56.557018+0000","last_peered":"2024-09-13T09:27:56.557018+0000","last_clean":"2024-09-13T09:27:56.557018+0000","last_became_active":"2024-09-13T09:27:39.355967+0000","last_became_peered":"2024-09-13T09:27:39.355967+0000","last_unstale":"2024-09-13T09:27:56.557018+0000","last_undegraded":"2024-09-13T09:27:56.557018+0000","last_fullsized":"2024-09-13T09:27:56.557018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:08.064206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":60092913,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":13,"num_read":9679,"num_read_kb":29818,"num_write":1271,"num_write_kb":11245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":10989136,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1535259958,"num_objects":1772,"num_object_clones":342,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":264,"num_read":212855,"num_read_kb":659978,"num_write":26969,"num_write_kb":397055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3082,"num_bytes_recovered":211673483,"num_keys_recovered":658,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81969152,"data_stored":472534405,"data_compressed":40415923,"data_compressed_allocated":45731840,"data_compressed_original":440234151,"omap_allocated":8553,"internal_metadata":0},"log_size":20014,"ondisk_log_size":20014,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70016,"kb_used_data":16760,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94301824,"statfs":{"total":96636764160,"available":96565067776,"internally_reserved":0,"allocated":17162240,"data_stored":93311690,"data_compressed":7952238,"data_compressed_allocated":8224768,"data_compressed_original":85614719,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.78300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.224,"5min":1.256,"15min":1.256},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.76900000000000002,"15min":0.75800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.875,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.2649999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.099},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1910000000000001,"15min":1.2529999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.484,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96199999999999997,"15min":0.97999999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.875},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.175,"5min":1.357,"15min":1.357},"last":1.1459999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.0409999999999999,"15min":1.0649999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.125},{"interface":"front","average":{"1min":0.89900000000000002,"5min":1.0029999999999999,"15min":1.0209999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.90000000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44316,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327524,"statfs":{"total":96636764160,"available":96591384576,"internally_reserved":0,"allocated":5058560,"data_stored":18964197,"data_compressed":1904445,"data_compressed_allocated":2068480,"data_compressed_original":17003491,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.86899999999999999,"15min":0.89700000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1100000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0800000000000001,"5min":1.173,"15min":1.173},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.92500000000000004,"15min":0.96399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.161,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1599999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.258}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1439999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.997,"15min":1.0289999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.161,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1970000000000001,"5min":5.149,"15min":5.149},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2490000000000001,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.387}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89600000000000002,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.181,"5min":2.032,"15min":2.032},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0980000000000001,"15min":1.119},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2509999999999999,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":1.048}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2949999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.3280000000000001,"15min":1.3939999999999999},"min":{"1min":0.59699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.4990000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84720,"kb_used_data":13416,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94287120,"statfs":{"total":96636764160,"available":96550010880,"internally_reserved":0,"allocated":13737984,"data_stored":68967529,"data_compressed":6019523,"data_compressed_allocated":6807552,"data_compressed_original":63318073,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.81499999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1160000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.496,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0920000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.081,"5min":1.083,"15min":1.083},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1539999999999999,"5min":1.242,"15min":1.242},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.503,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.393,"5min":1.554,"15min":1.554},"last":1.1879999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.94699999999999995,"15min":0.97099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3979999999999999,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.97499999999999998,"15min":0.98899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.617,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":0.99399999999999999},"min":{"1min":0.52400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.23},{"interface":"front","average":{"1min":0.996,"5min":1.091,"15min":1.107},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6000000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.873}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63252,"kb_used_data":18124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308588,"statfs":{"total":96636764160,"available":96571994112,"internally_reserved":0,"allocated":18558976,"data_stored":100313080,"data_compressed":9368840,"data_compressed_allocated":10174464,"data_compressed_original":93432607,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.52300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.274,"5min":1.276,"15min":1.276},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.143,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.92400000000000004},"min":{"1min":0.57899999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.2050000000000001,"5min":1.298,"15min":1.298},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.56000000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.234,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.443},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.351,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.1799999999999999,"5min":1.304,"15min":1.304},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96499999999999997,"15min":0.97299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.381,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.038,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.452,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.97699999999999998,"15min":0.97999999999999998},"min":{"1min":0.63800000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4139999999999999,"5min":1.605,"15min":1.605},"last":0.69899999999999995}]}]},{"osd":3,"up_from":74,"seq":317827579916,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80988,"kb_used_data":11988,"kb_used_omap":14,"kb_used_meta":68977,"kb_avail":94290852,"statfs":{"total":96636764160,"available":96553832448,"internally_reserved":0,"allocated":12275712,"data_stored":66270059,"data_compressed":4701353,"data_compressed_allocated":5734400,"data_compressed_original":60846168,"omap_allocated":14794,"internal_metadata":70633014},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38656,"kb_used_data":7544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333184,"statfs":{"total":96636764160,"available":96597180416,"internally_reserved":0,"allocated":7725056,"data_stored":34817629,"data_compressed":2581559,"data_compressed_allocated":3407872,"data_compressed_original":31549033,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.089,"5min":1.484,"15min":1.484},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89800000000000002,"5min":1.335,"15min":1.335},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.98899999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2310000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.98199999999999998,"15min":1.0009999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3420000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.306,"5min":1.587,"15min":1.587},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0049999999999999,"15min":0.98299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.6879999999999997,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.028,"15min":1.036},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.488,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.1779999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.28,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0349999999999999,"15min":1.048},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3420000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0489999999999999,"15min":1.056},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5029999999999999,"5min":2.048,"15min":2.048},"last":1.129},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.169},"min":{"1min":0.69399999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.556,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.89000000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44392,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327448,"statfs":{"total":96636764160,"available":96591306752,"internally_reserved":0,"allocated":13594624,"data_stored":76297071,"data_compressed":6385127,"data_compressed_allocated":7409664,"data_compressed_original":71908720,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,28],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.78400000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":12.441000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.70299999999999996,"15min":0.72099999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.98499999999999999,"5min":1.333,"15min":1.333},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.014,"5min":1.153,"15min":1.153},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.85699999999999998,"15min":0.90300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91400000000000003,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.8540000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.555}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1950000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":2.7130000000000001},{"interface":"front","average":{"1min":3.133,"5min":1.4350000000000001,"15min":1.1519999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":49.905999999999999,"5min":49.905999999999999,"15min":49.905999999999999},"last":12.487}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.01,"15min":1.034},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2170000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95199999999999996,"15min":0.95799999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":8.0109999999999992}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":1.0449999999999999,"15min":1.079},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.371,"5min":1.704,"15min":1.704},"last":12.519},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.258,"5min":1.514,"15min":1.514},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.081,"15min":1.109},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2490000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":14.143000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.099,"15min":1.1259999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.7,"15min":1.7},"last":1.891}]}]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44600,"kb_used_data":5284,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327240,"statfs":{"total":96636764160,"available":96591093760,"internally_reserved":0,"allocated":5410816,"data_stored":20922130,"data_compressed":1605920,"data_compressed_allocated":2043904,"data_compressed_original":18509572,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.312},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.437,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.018,"5min":1.24,"15min":1.24},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.292,"5min":1.996,"15min":1.996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6379999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1910000000000001,"5min":1.214,"15min":1.214},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.623,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.522,"5min":1.651,"15min":1.651},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.96799999999999997,"15min":0.97599999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.26,"5min":1.448,"15min":1.448},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":0.97399999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3300000000000001,"5min":1.333,"15min":1.333},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.357,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1,"15min":1.002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12144640,"data_stored":75616188,"data_compressed":6385127,"data_compressed_allocated":7409664,"data_compressed_original":71908720,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6254592,"data_stored":33440486,"data_compressed":2542374,"data_compressed_allocated":3354624,"data_compressed_original":30843181,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3866624,"data_stored":20203385,"data_compressed":1605920,"data_compressed_allocated":2043904,"data_compressed_original":18509572,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10858496,"data_stored":65601401,"data_compressed":4701353,"data_compressed_allocated":5734400,"data_compressed_original":60846168,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17158144,"data_stored":99194844,"data_compressed":9347217,"data_compressed_allocated":10145792,"data_compressed_original":92964371,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12365824,"data_stored":68321133,"data_compressed":6019523,"data_compressed_allocated":6807552,"data_compressed_original":63318073,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3629056,"data_stored":18093249,"data_compressed":1895178,"data_compressed_allocated":2056192,"data_compressed_original":16802787,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15691776,"data_stored":92063719,"data_compressed":7919231,"data_compressed_allocated":8179712,"data_compressed_original":85041279,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:27:59.020 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:27:59.076 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:27:59.072+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:27:59.098 INFO:teuthology.orchestra.run.smithi063.stdout:5743 write 0xa1c13ba thru 0xa1cb0f5 (0x9d3c bytes) 2024-09-13T09:27:59.112 INFO:teuthology.orchestra.run.smithi063.stdout:5744 clone 43 order 23 su 524288 sc 3 2024-09-13T09:27:59.170 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:59.170 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:59.177 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:59.172+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:59.179 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:27:59.278 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:27:59.272+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:27:59.325 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:59.326 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:59.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:27:59.379 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:27:59.372+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:27:59.382 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:59.385 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:27:59.398 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-13T09:27:58.554084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1535850345,"num_objects":1776,"num_object_clones":342,"num_object_copies":3552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1776,"num_whiteouts":264,"num_read":212971,"num_read_kb":660078,"num_write":27130,"num_write_kb":399473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3084,"num_bytes_recovered":211673521,"num_keys_recovered":658,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20134,"ondisk_log_size":20134,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":470940,"kb_used_data":91332,"kb_used_omap":25,"kb_used_meta":379366,"kb_avail":754503780,"statfs":{"total":773094113280,"available":772611870720,"internally_reserved":0,"allocated":93523968,"data_stored":479863385,"data_compressed":40519005,"data_compressed_allocated":45871104,"data_compressed_original":442182383,"omap_allocated":25946,"internal_metadata":388471462},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,28],"upper_bound":4},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11397597,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":13616,"num_read_kb":12079,"num_write":5171,"num_write_kb":9678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001065"},"pg_stats":[{"pgid":"3.1f","version":"121'430","reported_seq":1728,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.403356+0000","last_change":"2024-09-13T09:26:39.600998+0000","last_active":"2024-09-13T09:27:56.403356+0000","last_peered":"2024-09-13T09:27:56.403356+0000","last_clean":"2024-09-13T09:27:56.403356+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:27:56.403356+0000","last_undegraded":"2024-09-13T09:27:56.403356+0000","last_fullsized":"2024-09-13T09:27:56.403356+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":41177473,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":11,"num_read":2987,"num_read_kb":15929,"num_write":464,"num_write_kb":7559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"121'679","reported_seq":5271,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.486612+0000","last_change":"2024-09-13T09:27:05.744365+0000","last_active":"2024-09-13T09:27:55.486612+0000","last_peered":"2024-09-13T09:27:55.486612+0000","last_clean":"2024-09-13T09:27:55.486612+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:27:55.486612+0000","last_undegraded":"2024-09-13T09:27:55.486612+0000","last_fullsized":"2024-09-13T09:27:55.486612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":34456411,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":4,"num_read":6494,"num_read_kb":13244,"num_write":1002,"num_write_kb":6773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"122'460","reported_seq":4057,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.558900+0000","last_change":"2024-09-13T09:27:39.353427+0000","last_active":"2024-09-13T09:27:56.558900+0000","last_peered":"2024-09-13T09:27:56.558900+0000","last_clean":"2024-09-13T09:27:56.558900+0000","last_became_active":"2024-09-13T09:27:39.352330+0000","last_became_peered":"2024-09-13T09:27:39.352330+0000","last_unstale":"2024-09-13T09:27:56.558900+0000","last_undegraded":"2024-09-13T09:27:56.558900+0000","last_fullsized":"2024-09-13T09:27:56.558900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:56.346789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":57790951,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":9,"num_read":4071,"num_read_kb":20268,"num_write":541,"num_write_kb":11276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":12545300,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"122'506","reported_seq":5119,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.561886+0000","last_change":"2024-09-13T09:27:38.348001+0000","last_active":"2024-09-13T09:27:56.561886+0000","last_peered":"2024-09-13T09:27:56.561886+0000","last_clean":"2024-09-13T09:27:56.561886+0000","last_became_active":"2024-09-13T09:27:38.346735+0000","last_became_peered":"2024-09-13T09:27:38.346735+0000","last_unstale":"2024-09-13T09:27:56.561886+0000","last_undegraded":"2024-09-13T09:27:56.561886+0000","last_fullsized":"2024-09-13T09:27:56.561886+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:23.094925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":67602354,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":8,"num_read":4956,"num_read_kb":17791,"num_write":644,"num_write_kb":13016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":15156139,"num_keys_recovered":138,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"122'480","reported_seq":2899,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501631+0000","last_change":"2024-09-13T09:27:40.167421+0000","last_active":"2024-09-13T09:27:56.501631+0000","last_peered":"2024-09-13T09:27:56.501631+0000","last_clean":"2024-09-13T09:27:56.501631+0000","last_became_active":"2024-09-13T09:27:40.165702+0000","last_became_peered":"2024-09-13T09:27:40.165702+0000","last_unstale":"2024-09-13T09:27:56.501631+0000","last_undegraded":"2024-09-13T09:27:56.501631+0000","last_fullsized":"2024-09-13T09:27:56.501631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:21:08.885741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201516,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":12,"num_read":3733,"num_read_kb":10627,"num_write":552,"num_write_kb":6214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9930624,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"121'550","reported_seq":11532,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.568142+0000","last_change":"2024-09-13T09:27:37.336790+0000","last_active":"2024-09-13T09:27:56.568142+0000","last_peered":"2024-09-13T09:27:56.568142+0000","last_clean":"2024-09-13T09:27:56.568142+0000","last_became_active":"2024-09-13T09:27:37.334754+0000","last_became_peered":"2024-09-13T09:27:37.334754+0000","last_unstale":"2024-09-13T09:27:56.568142+0000","last_undegraded":"2024-09-13T09:27:56.568142+0000","last_fullsized":"2024-09-13T09:27:56.568142+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:54.865010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":46971248,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":6,"num_read":12925,"num_read_kb":21496,"num_write":634,"num_write_kb":6617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":8205452,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"121'507","reported_seq":2623,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.415353+0000","last_change":"2024-09-13T09:27:05.741503+0000","last_active":"2024-09-13T09:27:56.415353+0000","last_peered":"2024-09-13T09:27:56.415353+0000","last_clean":"2024-09-13T09:27:56.415353+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:27:56.415353+0000","last_undegraded":"2024-09-13T09:27:56.415353+0000","last_fullsized":"2024-09-13T09:27:56.415353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":29443717,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":7,"num_read":3351,"num_read_kb":9986,"num_write":535,"num_write_kb":8021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"121'490","reported_seq":1666,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.614985+0000","last_change":"2024-09-13T09:27:03.781188+0000","last_active":"2024-09-13T09:27:55.614985+0000","last_peered":"2024-09-13T09:27:55.614985+0000","last_clean":"2024-09-13T09:27:55.614985+0000","last_became_active":"2024-09-13T09:27:03.711948+0000","last_became_peered":"2024-09-13T09:27:03.711948+0000","last_unstale":"2024-09-13T09:27:55.614985+0000","last_undegraded":"2024-09-13T09:27:55.614985+0000","last_fullsized":"2024-09-13T09:27:55.614985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:42.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":29701076,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":6,"num_read":3304,"num_read_kb":15167,"num_write":512,"num_write_kb":13675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":8502479,"num_keys_recovered":39,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"121'609","reported_seq":1524,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.601987+0000","last_change":"2024-09-13T09:26:42.575210+0000","last_active":"2024-09-13T09:27:55.601987+0000","last_peered":"2024-09-13T09:27:55.601987+0000","last_clean":"2024-09-13T09:27:55.601987+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:27:55.601987+0000","last_undegraded":"2024-09-13T09:27:55.601987+0000","last_fullsized":"2024-09-13T09:27:55.601987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:47.404587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10535024,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":5,"num_read":3079,"num_read_kb":4895,"num_write":723,"num_write_kb":3524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"122'441","reported_seq":5705,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.522068+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:27:56.522068+0000","last_peered":"2024-09-13T09:27:56.522068+0000","last_clean":"2024-09-13T09:27:56.522068+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:27:56.522068+0000","last_undegraded":"2024-09-13T09:27:56.522068+0000","last_fullsized":"2024-09-13T09:27:56.522068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":53405795,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":8,"num_read":5562,"num_read_kb":29365,"num_write":556,"num_write_kb":16204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"121'387","reported_seq":7499,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417871+0000","last_change":"2024-09-13T09:27:05.750137+0000","last_active":"2024-09-13T09:27:56.417871+0000","last_peered":"2024-09-13T09:27:56.417871+0000","last_clean":"2024-09-13T09:27:56.417871+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:27:56.417871+0000","last_undegraded":"2024-09-13T09:27:56.417871+0000","last_fullsized":"2024-09-13T09:27:56.417871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":26932203,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":6,"num_read":7428,"num_read_kb":26689,"num_write":415,"num_write_kb":8419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"121'429","reported_seq":5544,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.087395+0000","last_change":"2024-09-13T09:27:37.337476+0000","last_active":"2024-09-13T09:27:56.087395+0000","last_peered":"2024-09-13T09:27:56.087395+0000","last_clean":"2024-09-13T09:27:56.087395+0000","last_became_active":"2024-09-13T09:27:37.335199+0000","last_became_peered":"2024-09-13T09:27:37.335199+0000","last_unstale":"2024-09-13T09:27:56.087395+0000","last_undegraded":"2024-09-13T09:27:56.087395+0000","last_fullsized":"2024-09-13T09:27:56.087395+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:24:06.331221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":46596591,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":8,"num_read":5711,"num_read_kb":12853,"num_write":460,"num_write_kb":5527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":4959398,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":132,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501625+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:27:56.501625+0000","last_peered":"2024-09-13T09:27:56.501625+0000","last_clean":"2024-09-13T09:27:56.501625+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:27:56.501625+0000","last_undegraded":"2024-09-13T09:27:56.501625+0000","last_fullsized":"2024-09-13T09:27:56.501625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"121'420","reported_seq":1244,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.415720+0000","last_change":"2024-09-13T09:27:05.753951+0000","last_active":"2024-09-13T09:27:56.415720+0000","last_peered":"2024-09-13T09:27:56.415720+0000","last_clean":"2024-09-13T09:27:56.415720+0000","last_became_active":"2024-09-13T09:27:05.753131+0000","last_became_peered":"2024-09-13T09:27:05.753131+0000","last_unstale":"2024-09-13T09:27:56.415720+0000","last_undegraded":"2024-09-13T09:27:56.415720+0000","last_fullsized":"2024-09-13T09:27:56.415720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:21.776084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":69325118,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":8,"num_read":1004,"num_read_kb":6232,"num_write":503,"num_write_kb":8017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501774+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:27:56.501774+0000","last_peered":"2024-09-13T09:27:56.501774+0000","last_clean":"2024-09-13T09:27:56.501774+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:27:56.501774+0000","last_undegraded":"2024-09-13T09:27:56.501774+0000","last_fullsized":"2024-09-13T09:27:56.501774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"121'710","reported_seq":3200,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.395622+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:27:56.395622+0000","last_peered":"2024-09-13T09:27:56.395622+0000","last_clean":"2024-09-13T09:27:56.395622+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:27:56.395622+0000","last_undegraded":"2024-09-13T09:27:56.395622+0000","last_fullsized":"2024-09-13T09:27:56.395622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":35108989,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":6,"num_read":5051,"num_read_kb":18443,"num_write":993,"num_write_kb":21566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.481556+0000","last_change":"2024-09-13T09:27:34.289812+0000","last_active":"2024-09-13T09:27:55.481556+0000","last_peered":"2024-09-13T09:27:55.481556+0000","last_clean":"2024-09-13T09:27:55.481556+0000","last_became_active":"2024-09-13T09:27:34.289466+0000","last_became_peered":"2024-09-13T09:27:34.289466+0000","last_unstale":"2024-09-13T09:27:55.481556+0000","last_undegraded":"2024-09-13T09:27:55.481556+0000","last_fullsized":"2024-09-13T09:27:55.481556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:42.497510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"121'652","reported_seq":5586,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.486868+0000","last_change":"2024-09-13T09:27:05.741382+0000","last_active":"2024-09-13T09:27:55.486868+0000","last_peered":"2024-09-13T09:27:55.486868+0000","last_clean":"2024-09-13T09:27:55.486868+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:27:55.486868+0000","last_undegraded":"2024-09-13T09:27:55.486868+0000","last_fullsized":"2024-09-13T09:27:55.486868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":74457018,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":9,"num_read":7238,"num_read_kb":25922,"num_write":963,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:54.472800+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:27:54.472800+0000","last_peered":"2024-09-13T09:27:54.472800+0000","last_clean":"2024-09-13T09:27:54.472800+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:27:54.472800+0000","last_undegraded":"2024-09-13T09:27:54.472800+0000","last_fullsized":"2024-09-13T09:27:54.472800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"121'689","reported_seq":5426,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.411416+0000","last_change":"2024-09-13T09:26:42.570158+0000","last_active":"2024-09-13T09:27:56.411416+0000","last_peered":"2024-09-13T09:27:56.411416+0000","last_clean":"2024-09-13T09:27:56.411416+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:27:56.411416+0000","last_undegraded":"2024-09-13T09:27:56.411416+0000","last_fullsized":"2024-09-13T09:27:56.411416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53785345,"num_objects":66,"num_object_clones":12,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":10,"num_read":7565,"num_read_kb":24285,"num_write":904,"num_write_kb":8412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"121'1254","reported_seq":2685,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417251+0000","last_change":"2024-09-13T09:27:40.165078+0000","last_active":"2024-09-13T09:27:56.417251+0000","last_peered":"2024-09-13T09:27:56.417251+0000","last_clean":"2024-09-13T09:27:56.417251+0000","last_became_active":"2024-09-13T09:27:40.164156+0000","last_became_peered":"2024-09-13T09:27:40.164156+0000","last_unstale":"2024-09-13T09:27:56.417251+0000","last_undegraded":"2024-09-13T09:27:56.417251+0000","last_fullsized":"2024-09-13T09:27:56.417251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:58.977950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13597679,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":6,"num_read":6916,"num_read_kb":15309,"num_write":1974,"num_write_kb":6812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":6885568,"num_keys_recovered":105,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":263,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490814+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:27:55.490814+0000","last_peered":"2024-09-13T09:27:55.490814+0000","last_clean":"2024-09-13T09:27:55.490814+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:27:55.490814+0000","last_undegraded":"2024-09-13T09:27:55.490814+0000","last_fullsized":"2024-09-13T09:27:55.490814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":119,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:54.482791+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:27:54.482791+0000","last_peered":"2024-09-13T09:27:54.482791+0000","last_clean":"2024-09-13T09:27:54.482791+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:27:54.482791+0000","last_undegraded":"2024-09-13T09:27:54.482791+0000","last_fullsized":"2024-09-13T09:27:54.482791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"121'889","reported_seq":8398,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.080626+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:27:56.080626+0000","last_peered":"2024-09-13T09:27:56.080626+0000","last_clean":"2024-09-13T09:27:56.080626+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:27:56.080626+0000","last_undegraded":"2024-09-13T09:27:56.080626+0000","last_fullsized":"2024-09-13T09:27:56.080626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":64588489,"num_objects":58,"num_object_clones":10,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":9,"num_read":10690,"num_read_kb":23133,"num_write":1323,"num_write_kb":18418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":145,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490844+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:27:55.490844+0000","last_peered":"2024-09-13T09:27:55.490844+0000","last_clean":"2024-09-13T09:27:55.490844+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:27:55.490844+0000","last_undegraded":"2024-09-13T09:27:55.490844+0000","last_fullsized":"2024-09-13T09:27:55.490844+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"121'539","reported_seq":10201,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417876+0000","last_change":"2024-09-13T09:27:38.350122+0000","last_active":"2024-09-13T09:27:56.417876+0000","last_peered":"2024-09-13T09:27:56.417876+0000","last_clean":"2024-09-13T09:27:56.417876+0000","last_became_active":"2024-09-13T09:27:38.348807+0000","last_became_peered":"2024-09-13T09:27:38.348807+0000","last_unstale":"2024-09-13T09:27:56.417876+0000","last_undegraded":"2024-09-13T09:27:56.417876+0000","last_fullsized":"2024-09-13T09:27:56.417876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:08.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":86966958,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":11,"num_read":9772,"num_read_kb":38350,"num_write":673,"num_write_kb":31494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":37856068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":136,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.497662+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:27:55.497662+0000","last_peered":"2024-09-13T09:27:55.497662+0000","last_clean":"2024-09-13T09:27:55.497662+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:27:55.497662+0000","last_undegraded":"2024-09-13T09:27:55.497662+0000","last_fullsized":"2024-09-13T09:27:55.497662+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"121'1861","reported_seq":6509,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.694854+0000","last_change":"2024-09-13T09:27:05.745925+0000","last_active":"2024-09-13T09:27:55.694854+0000","last_peered":"2024-09-13T09:27:55.694854+0000","last_clean":"2024-09-13T09:27:55.694854+0000","last_became_active":"2024-09-13T09:27:05.744803+0000","last_became_peered":"2024-09-13T09:27:05.744803+0000","last_unstale":"2024-09-13T09:27:55.694854+0000","last_undegraded":"2024-09-13T09:27:55.694854+0000","last_fullsized":"2024-09-13T09:27:55.694854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1861,"log_dups_size":0,"ondisk_log_size":1861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:40.185419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":25584455,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":5,"num_read":13735,"num_read_kb":16194,"num_write":3351,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":135,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490772+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:27:55.490772+0000","last_peered":"2024-09-13T09:27:55.490772+0000","last_clean":"2024-09-13T09:27:55.490772+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:27:55.490772+0000","last_undegraded":"2024-09-13T09:27:55.490772+0000","last_fullsized":"2024-09-13T09:27:55.490772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"120'486","reported_seq":1871,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.460423+0000","last_change":"2024-09-13T09:27:05.745269+0000","last_active":"2024-09-13T09:27:55.460423+0000","last_peered":"2024-09-13T09:27:55.460423+0000","last_clean":"2024-09-13T09:27:55.460423+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:27:55.460423+0000","last_undegraded":"2024-09-13T09:27:55.460423+0000","last_fullsized":"2024-09-13T09:27:55.460423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":23606372,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":9,"num_read":2940,"num_read_kb":13676,"num_write":552,"num_write_kb":6739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"121'398","reported_seq":11067,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417645+0000","last_change":"2024-09-13T09:27:39.357803+0000","last_active":"2024-09-13T09:27:56.417645+0000","last_peered":"2024-09-13T09:27:56.417645+0000","last_clean":"2024-09-13T09:27:56.417645+0000","last_became_active":"2024-09-13T09:27:39.355670+0000","last_became_peered":"2024-09-13T09:27:39.355670+0000","last_unstale":"2024-09-13T09:27:56.417645+0000","last_undegraded":"2024-09-13T09:27:56.417645+0000","last_fullsized":"2024-09-13T09:27:56.417645+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:18.749072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":55902485,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":10,"num_read":10451,"num_read_kb":21041,"num_write":430,"num_write_kb":14364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":9133614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"121'382","reported_seq":3629,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.112269+0000","last_change":"2024-09-13T09:27:39.354051+0000","last_active":"2024-09-13T09:27:56.112269+0000","last_peered":"2024-09-13T09:27:56.112269+0000","last_clean":"2024-09-13T09:27:56.112269+0000","last_became_active":"2024-09-13T09:27:39.352709+0000","last_became_peered":"2024-09-13T09:27:39.352709+0000","last_unstale":"2024-09-13T09:27:56.112269+0000","last_undegraded":"2024-09-13T09:27:56.112269+0000","last_fullsized":"2024-09-13T09:27:56.112269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:01.998737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":38529235,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":9,"num_read":3008,"num_read_kb":20333,"num_write":391,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":5132540,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"121'512","reported_seq":4076,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.633966+0000","last_change":"2024-09-13T09:27:05.743332+0000","last_active":"2024-09-13T09:27:55.633966+0000","last_peered":"2024-09-13T09:27:55.633966+0000","last_clean":"2024-09-13T09:27:55.633966+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:27:55.633966+0000","last_undegraded":"2024-09-13T09:27:55.633966+0000","last_fullsized":"2024-09-13T09:27:55.633966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":68568851,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":10,"num_read":3792,"num_read_kb":27296,"num_write":599,"num_write_kb":19857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"121'527","reported_seq":7283,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.418174+0000","last_change":"2024-09-13T09:27:05.746491+0000","last_active":"2024-09-13T09:27:56.418174+0000","last_peered":"2024-09-13T09:27:56.418174+0000","last_clean":"2024-09-13T09:27:56.418174+0000","last_became_active":"2024-09-13T09:27:05.745595+0000","last_became_peered":"2024-09-13T09:27:05.745595+0000","last_unstale":"2024-09-13T09:27:56.418174+0000","last_undegraded":"2024-09-13T09:27:56.418174+0000","last_fullsized":"2024-09-13T09:27:56.418174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:38.773883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":42585420,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":4,"num_read":7699,"num_read_kb":21253,"num_write":745,"num_write_kb":15259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"121'398","reported_seq":14221,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.401322+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:27:56.401322+0000","last_peered":"2024-09-13T09:27:56.401322+0000","last_clean":"2024-09-13T09:27:56.401322+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:27:56.401322+0000","last_undegraded":"2024-09-13T09:27:56.401322+0000","last_fullsized":"2024-09-13T09:27:56.401322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":82298447,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":14,"num_read":13865,"num_read_kb":35794,"num_write":479,"num_write_kb":29504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"121'1075","reported_seq":8014,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417703+0000","last_change":"2024-09-13T09:27:40.166348+0000","last_active":"2024-09-13T09:27:56.417703+0000","last_peered":"2024-09-13T09:27:56.417703+0000","last_clean":"2024-09-13T09:27:56.417703+0000","last_became_active":"2024-09-13T09:27:40.164774+0000","last_became_peered":"2024-09-13T09:27:40.164774+0000","last_unstale":"2024-09-13T09:27:56.417703+0000","last_undegraded":"2024-09-13T09:27:56.417703+0000","last_fullsized":"2024-09-13T09:27:56.417703+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:43.498002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":77899202,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":12,"num_read":10748,"num_read_kb":23563,"num_write":1746,"num_write_kb":18987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":13609565,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"121'411","reported_seq":6266,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.202806+0000","last_change":"2024-09-13T09:27:37.336194+0000","last_active":"2024-09-13T09:27:56.202806+0000","last_peered":"2024-09-13T09:27:56.202806+0000","last_clean":"2024-09-13T09:27:56.202806+0000","last_became_active":"2024-09-13T09:27:37.335370+0000","last_became_peered":"2024-09-13T09:27:37.335370+0000","last_unstale":"2024-09-13T09:27:56.202806+0000","last_undegraded":"2024-09-13T09:27:56.202806+0000","last_fullsized":"2024-09-13T09:27:56.202806+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:20.967570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":59679891,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":6,"num_read":5598,"num_read_kb":29560,"num_write":462,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":13410696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"121'761","reported_seq":3605,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417618+0000","last_change":"2024-09-13T09:26:42.574072+0000","last_active":"2024-09-13T09:27:56.417618+0000","last_peered":"2024-09-13T09:27:56.417618+0000","last_clean":"2024-09-13T09:27:56.417618+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:27:56.417618+0000","last_undegraded":"2024-09-13T09:27:56.417618+0000","last_fullsized":"2024-09-13T09:27:56.417618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":37693572,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":10,"num_read":4846,"num_read_kb":23323,"num_write":1104,"num_write_kb":9498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"121'705","reported_seq":7674,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417713+0000","last_change":"2024-09-13T09:27:41.168314+0000","last_active":"2024-09-13T09:27:56.417713+0000","last_peered":"2024-09-13T09:27:56.417713+0000","last_clean":"2024-09-13T09:27:56.417713+0000","last_became_active":"2024-09-13T09:27:41.165699+0000","last_became_peered":"2024-09-13T09:27:41.165699+0000","last_unstale":"2024-09-13T09:27:56.417713+0000","last_undegraded":"2024-09-13T09:27:56.417713+0000","last_fullsized":"2024-09-13T09:27:56.417713+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:03.888324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":49286710,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":9,"num_read":9644,"num_read_kb":25247,"num_write":885,"num_write_kb":8751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":9211408,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"121'439","reported_seq":5275,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.242228+0000","last_change":"2024-09-13T09:27:17.867086+0000","last_active":"2024-09-13T09:27:56.242228+0000","last_peered":"2024-09-13T09:27:56.242228+0000","last_clean":"2024-09-13T09:27:56.242228+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:27:56.242228+0000","last_undegraded":"2024-09-13T09:27:56.242228+0000","last_fullsized":"2024-09-13T09:27:56.242228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57888450,"num_objects":55,"num_object_clones":7,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":4,"num_read":5013,"num_read_kb":22896,"num_write":583,"num_write_kb":14850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"122'938","reported_seq":7127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.557018+0000","last_change":"2024-09-13T09:27:39.358188+0000","last_active":"2024-09-13T09:27:56.557018+0000","last_peered":"2024-09-13T09:27:56.557018+0000","last_clean":"2024-09-13T09:27:56.557018+0000","last_became_active":"2024-09-13T09:27:39.355967+0000","last_became_peered":"2024-09-13T09:27:39.355967+0000","last_unstale":"2024-09-13T09:27:56.557018+0000","last_undegraded":"2024-09-13T09:27:56.557018+0000","last_fullsized":"2024-09-13T09:27:56.557018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:08.064206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":60092913,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":13,"num_read":9679,"num_read_kb":29818,"num_write":1271,"num_write_kb":11245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":10989136,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1535259958,"num_objects":1772,"num_object_clones":342,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":264,"num_read":212855,"num_read_kb":659978,"num_write":26969,"num_write_kb":397055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3082,"num_bytes_recovered":211673483,"num_keys_recovered":658,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81969152,"data_stored":472534405,"data_compressed":40415923,"data_compressed_allocated":45731840,"data_compressed_original":440234151,"omap_allocated":8553,"internal_metadata":0},"log_size":20014,"ondisk_log_size":20014,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70016,"kb_used_data":16760,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94301824,"statfs":{"total":96636764160,"available":96565067776,"internally_reserved":0,"allocated":17162240,"data_stored":93311690,"data_compressed":7952238,"data_compressed_allocated":8224768,"data_compressed_original":85614719,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.78300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.224,"5min":1.256,"15min":1.256},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.76900000000000002,"15min":0.75800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.875,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.2649999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.099},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1910000000000001,"15min":1.2529999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.484,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96199999999999997,"15min":0.97999999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.875},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.175,"5min":1.357,"15min":1.357},"last":1.1459999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.0409999999999999,"15min":1.0649999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.125},{"interface":"front","average":{"1min":0.89900000000000002,"5min":1.0029999999999999,"15min":1.0209999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.90000000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44316,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327524,"statfs":{"total":96636764160,"available":96591384576,"internally_reserved":0,"allocated":5058560,"data_stored":18964197,"data_compressed":1904445,"data_compressed_allocated":2068480,"data_compressed_original":17003491,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.86899999999999999,"15min":0.89700000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1100000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0800000000000001,"5min":1.173,"15min":1.173},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.92500000000000004,"15min":0.96399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.161,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1599999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.258}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1439999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.997,"15min":1.0289999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.161,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1970000000000001,"5min":5.149,"15min":5.149},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2490000000000001,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.387}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89600000000000002,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.181,"5min":2.032,"15min":2.032},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0980000000000001,"15min":1.119},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2509999999999999,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":1.048}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2949999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.3280000000000001,"15min":1.3939999999999999},"min":{"1min":0.59699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.4990000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84720,"kb_used_data":13416,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94287120,"statfs":{"total":96636764160,"available":96550010880,"internally_reserved":0,"allocated":13737984,"data_stored":68967529,"data_compressed":6019523,"data_compressed_allocated":6807552,"data_compressed_original":63318073,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.81499999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1160000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.496,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0920000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.081,"5min":1.083,"15min":1.083},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1539999999999999,"5min":1.242,"15min":1.242},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.503,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.393,"5min":1.554,"15min":1.554},"last":1.1879999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.94699999999999995,"15min":0.97099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3979999999999999,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.97499999999999998,"15min":0.98899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.617,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":0.99399999999999999},"min":{"1min":0.52400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.23},{"interface":"front","average":{"1min":0.996,"5min":1.091,"15min":1.107},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6000000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.873}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63252,"kb_used_data":18124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308588,"statfs":{"total":96636764160,"available":96571994112,"internally_reserved":0,"allocated":18558976,"data_stored":100313080,"data_compressed":9368840,"data_compressed_allocated":10174464,"data_compressed_original":93432607,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.52300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.274,"5min":1.276,"15min":1.276},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.143,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.92400000000000004},"min":{"1min":0.57899999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.2050000000000001,"5min":1.298,"15min":1.298},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.56000000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.234,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.443},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.351,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.1799999999999999,"5min":1.304,"15min":1.304},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96499999999999997,"15min":0.97299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.381,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.038,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.452,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.97699999999999998,"15min":0.97999999999999998},"min":{"1min":0.63800000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4139999999999999,"5min":1.605,"15min":1.605},"last":0.69899999999999995}]}]},{"osd":3,"up_from":74,"seq":317827579916,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80988,"kb_used_data":11988,"kb_used_omap":14,"kb_used_meta":68977,"kb_avail":94290852,"statfs":{"total":96636764160,"available":96553832448,"internally_reserved":0,"allocated":12275712,"data_stored":66270059,"data_compressed":4701353,"data_compressed_allocated":5734400,"data_compressed_original":60846168,"omap_allocated":14794,"internal_metadata":70633014},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38656,"kb_used_data":7544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333184,"statfs":{"total":96636764160,"available":96597180416,"internally_reserved":0,"allocated":7725056,"data_stored":34817629,"data_compressed":2581559,"data_compressed_allocated":3407872,"data_compressed_original":31549033,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.089,"5min":1.484,"15min":1.484},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89800000000000002,"5min":1.335,"15min":1.335},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.98899999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2310000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.98199999999999998,"15min":1.0009999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3420000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.306,"5min":1.587,"15min":1.587},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0049999999999999,"15min":0.98299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.6879999999999997,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.028,"15min":1.036},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.488,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.1779999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.28,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0349999999999999,"15min":1.048},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3420000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0489999999999999,"15min":1.056},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5029999999999999,"5min":2.048,"15min":2.048},"last":1.129},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.169},"min":{"1min":0.69399999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.556,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.89000000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44392,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327448,"statfs":{"total":96636764160,"available":96591306752,"internally_reserved":0,"allocated":13594624,"data_stored":76297071,"data_compressed":6385127,"data_compressed_allocated":7409664,"data_compressed_original":71908720,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,28],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.78400000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":12.441000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.70299999999999996,"15min":0.72099999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.98499999999999999,"5min":1.333,"15min":1.333},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.014,"5min":1.153,"15min":1.153},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.85699999999999998,"15min":0.90300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91400000000000003,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.8540000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.555}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1950000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":2.7130000000000001},{"interface":"front","average":{"1min":3.133,"5min":1.4350000000000001,"15min":1.1519999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":49.905999999999999,"5min":49.905999999999999,"15min":49.905999999999999},"last":12.487}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.01,"15min":1.034},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2170000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95199999999999996,"15min":0.95799999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":8.0109999999999992}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":1.0449999999999999,"15min":1.079},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.371,"5min":1.704,"15min":1.704},"last":12.519},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.258,"5min":1.514,"15min":1.514},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.081,"15min":1.109},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2490000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":14.143000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.099,"15min":1.1259999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.7,"15min":1.7},"last":1.891}]}]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44600,"kb_used_data":5284,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327240,"statfs":{"total":96636764160,"available":96591093760,"internally_reserved":0,"allocated":5410816,"data_stored":20922130,"data_compressed":1605920,"data_compressed_allocated":2043904,"data_compressed_original":18509572,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.312},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.437,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.018,"5min":1.24,"15min":1.24},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.292,"5min":1.996,"15min":1.996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6379999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1910000000000001,"5min":1.214,"15min":1.214},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.623,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.522,"5min":1.651,"15min":1.651},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.96799999999999997,"15min":0.97599999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.26,"5min":1.448,"15min":1.448},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":0.97399999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3300000000000001,"5min":1.333,"15min":1.333},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.357,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1,"15min":1.002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12144640,"data_stored":75616188,"data_compressed":6385127,"data_compressed_allocated":7409664,"data_compressed_original":71908720,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6254592,"data_stored":33440486,"data_compressed":2542374,"data_compressed_allocated":3354624,"data_compressed_original":30843181,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3866624,"data_stored":20203385,"data_compressed":1605920,"data_compressed_allocated":2043904,"data_compressed_original":18509572,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10858496,"data_stored":65601401,"data_compressed":4701353,"data_compressed_allocated":5734400,"data_compressed_original":60846168,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17158144,"data_stored":99194844,"data_compressed":9347217,"data_compressed_allocated":10145792,"data_compressed_original":92964371,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12365824,"data_stored":68321133,"data_compressed":6019523,"data_compressed_allocated":6807552,"data_compressed_original":63318073,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3629056,"data_stored":18093249,"data_compressed":1895178,"data_compressed_allocated":2056192,"data_compressed_original":16802787,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15691776,"data_stored":92063719,"data_compressed":7919231,"data_compressed_allocated":8179712,"data_compressed_original":85041279,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:27:59.401 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:27:59.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:27:59.401 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:27:59.401 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:27:59.452 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:59.452 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:59.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:27:59.480 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:59.474+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:59.536 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:27:59.574 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:59.575 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:59.581 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:27:59.574+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:27:59.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:27:59.675 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone42 from 0xa1cb0f6 (overlap 0xd42efc) to 0x3fa7b2 2024-09-13T09:27:59.682 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:59.678+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:59.707 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:59.707 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:59.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:27:59.782 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:27:59.778+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:27:59.807 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:27:59.807 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:27:59.807 INFO:teuthology.orchestra.run.smithi063.stdout: "epoch": 125, 2024-09-13T09:27:59.807 INFO:teuthology.orchestra.run.smithi063.stdout: "fsid": "eb7f03c3-9691-4766-8999-6f4f526cda71", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "created": "2024-09-13T09:25:30.379173+0000", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "modified": "2024-09-13T09:27:59.517478+0000", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "last_up_change": "2024-09-13T09:27:01.686116+0000", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "last_in_change": "2024-09-13T09:27:33.261291+0000", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_num": 5804032, 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_set": [ 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "nodeep-scrub", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "pglog_hardlimit", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "purged_snapdirs", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "recovery_deletes", 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: "sortbitwise" 2024-09-13T09:27:59.808 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_version": 6, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_snapshot": "", 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_max": 3, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "max_osd": 8, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "min_compat_client": "jewel", 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "require_osd_release": "reef", 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "allow_crimson": false, 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: "pools": [ 2024-09-13T09:27:59.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 1, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "rbd", 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:41.357433+0000", 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:59.810 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 8, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 8, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 8, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 8, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 8, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:59.811 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 3, 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 19, 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:59.812 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:59.813 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:59.814 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 2, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 2, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 1, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:59.815 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 2, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": ".mgr", 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:42.578616+0000", 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 1, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool", 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:59.816 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 1, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 1, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 1, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 1, 2024-09-13T09:27:59.817 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 1, 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "19", 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:59.818 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 0, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 0, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:59.819 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:59.820 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "options": { 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_max": 32, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_min": 1 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.821 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": {} 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 8, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 8, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 0.25, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 2, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 2, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "pool": 3, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_name": "pool_client.0", 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "create_time": "2024-09-13T09:25:51.228779+0000", 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flags": 8193, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type": 1, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 2, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "min_size": 1, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_rule": 0, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:27:59.823 INFO:teuthology.orchestra.run.smithi063.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "object_hash": 2, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num": 32, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num": 32, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_placement_num_target": 32, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_target": 32, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_num_pending": 32, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "last_pg_merge_meta": { 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "source_pgid": "0.0", 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "ready_epoch": 0, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_started": 0, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "last_epoch_clean": 0, 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "source_version": "0'0", 2024-09-13T09:27:59.824 INFO:teuthology.orchestra.run.smithi063.stdout: "target_version": "0'0" 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "last_change": "125", 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend": "0", 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "auid": 0, 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_seq": 46, 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "snap_epoch": 125, 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "pool_snaps": [], 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps": "[]", 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_bytes": 0, 2024-09-13T09:27:59.825 INFO:teuthology.orchestra.run.smithi063.stdout: "quota_max_objects": 0, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tiers": [], 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "tier_of": -1, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "read_tier": -1, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "write_tier": -1, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_mode": "none", 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_bytes": 0, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "target_max_objects": 0, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_flush_age": 0, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "cache_min_evict_age": 0, 2024-09-13T09:27:59.826 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profile": "", 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_params": { 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "none" 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_period": 0, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_count": 0, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "use_gmt_hitset": true, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:27:59.827 INFO:teuthology.orchestra.run.smithi063.stdout: "grade_table": [], 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "stripe_width": 0, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "expected_num_objects": 0, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "fast_read": false, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "options": {}, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "application_metadata": { 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "rbd": {} 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "read_balance": { 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "score_acting": 1.5, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "score_stable": 1.5, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "optimal_score": 1, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_acting": 1.5, 2024-09-13T09:27:59.828 INFO:teuthology.orchestra.run.smithi063.stdout: "raw_score_stable": 1.5, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity": 1, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osds": [ 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "54ea2bd8-55b0-4ded-8a16-041261c5ba5a", 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.829 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 108, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6816", 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6817", 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6818", 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6819", 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6822", 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6823", 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6820", 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6821", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2906107514 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6817/2906107514", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6819/2906107514", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6823/2906107514", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6821/2906107514", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.834 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "0c95aee9-f871-4f6f-bbbd-1206db5d331c", 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:59.835 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 104, 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6824", 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6825", 2024-09-13T09:27:59.836 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6826", 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6827", 2024-09-13T09:27:59.837 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6830", 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6831", 2024-09-13T09:27:59.838 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6828", 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.839 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6829", 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3622854518 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6825/3622854518", 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6827/3622854518", 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6831/3622854518", 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6829/3622854518", 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.840 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "f8253db2-d4da-46b5-95cf-cd5c9ebd7328", 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 14, 2024-09-13T09:27:59.841 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 106, 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6800", 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6801", 2024-09-13T09:27:59.842 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6802", 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6803", 2024-09-13T09:27:59.843 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6806", 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.844 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6807", 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6804", 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.845 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6805", 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2105659441 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6801/2105659441", 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6803/2105659441", 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6807/2105659441", 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6805/2105659441", 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "67c11a52-baeb-4ea8-89ea-26018ffdc35a", 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 15, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 46, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 74, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 77, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 48, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.847 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6808", 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6809", 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6810", 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6811", 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.849 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6814", 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6815", 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6812", 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.63:6813", 2024-09-13T09:27:59.851 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4293004354 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.63:6809/4293004354", 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.63:6811/4293004354", 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.63:6815/4293004354", 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.63:6813/4293004354", 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "1cec9702-fa09-4d7c-b111-7d10b1950cfe", 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 107, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.853 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6800", 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6801", 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.854 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6802", 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6803", 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6806", 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6807", 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.856 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6804", 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6805", 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 1789187713 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.857 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6801/1789187713", 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6803/1789187713", 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6807/1789187713", 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6805/1789187713", 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "7c0b908a-65c9-426c-867c-cc8ed65084b3", 2024-09-13T09:27:59.858 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 107, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.859 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6809", 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6811", 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6813", 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6815", 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.861 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6821", 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6823", 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.862 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6817", 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6819", 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 3410135021 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.863 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6811/3410135021", 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6815/3410135021", 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3410135021", 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6819/3410135021", 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "78d5990a-4535-4110-ba04-88f01ab9fe28", 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.864 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 107, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.865 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6808", 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6810", 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.866 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6812", 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6814", 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.867 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6820", 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6822", 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6816", 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6818", 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 4152410871 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6810/4152410871", 2024-09-13T09:27:59.869 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6814/4152410871", 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6822/4152410871", 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6818/4152410871", 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "uuid": "53f8511c-729f-4423-8eac-813c01f8f72b", 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "up": 1, 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "in": 1, 2024-09-13T09:27:59.870 INFO:teuthology.orchestra.run.smithi063.stdout: "weight": 1, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_affinity": 1, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_begin": 0, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "last_clean_end": 0, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "up_from": 15, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "up_thru": 109, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "down_at": 0, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "lost_at": 0, 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addrs": { 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6824", 2024-09-13T09:27:59.871 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6825", 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addrs": { 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.872 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6826", 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6827", 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addrs": { 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6830", 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6831", 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addrs": { 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: "addrvec": [ 2024-09-13T09:27:59.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v2", 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6828", 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "v1", 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "addr": "172.21.15.99:6829", 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "nonce": 2923799929 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "public_addr": "172.21.15.99:6825/2923799929", 2024-09-13T09:27:59.875 INFO:teuthology.orchestra.run.smithi063.stdout: "cluster_addr": "172.21.15.99:6827/2923799929", 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2923799929", 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2923799929", 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "state": [ 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "exists", 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "up" 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "osd_xinfo": [ 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 0, 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.876 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.267895+0000", 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 1, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.161313+0000", 2024-09-13T09:27:59.877 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 2, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.447964+0000", 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 3, 2024-09-13T09:27:59.878 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:38.021320+0000", 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 48 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 4, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.879 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.883961+0000", 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 5, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.880 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.938146+0000", 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 6, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.627237+0000", 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": 7, 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "down_stamp": "0.000000", 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_probability": 0, 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "laggy_interval": 0, 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "features": 4540138322906710015, 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "old_weight": 0, 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "last_purged_snaps_scrub": "2024-09-13T09:25:37.887523+0000", 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "dead_epoch": 0 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap": [], 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_items": [], 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_upmap_primaries": [], 2024-09-13T09:27:59.882 INFO:teuthology.orchestra.run.smithi063.stdout: "pg_temp": [], 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "primary_temp": [], 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "blocklist": {}, 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "range_blocklist": {}, 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "erasure_code_profiles": { 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "default": { 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "crush-failure-domain": "osd", 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "k": "2", 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "m": "1", 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "plugin": "jerasure", 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "technique": "reed_sol_van" 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "removed_snaps_queue": [], 2024-09-13T09:27:59.883 INFO:teuthology.orchestra.run.smithi063.stdout: "new_removed_snaps": [], 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "new_purged_snaps": [], 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "crush_node_flags": {}, 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "device_class_flags": {}, 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode": { 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_enabled": false, 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_bucket_count": 0, 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: "stretch_mode_bucket": 0 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:27:59.884 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:27:59.886 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:27:59.886 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:27:59.887 INFO:tasks.thrashosds.thrasher:j is {'epoch': 125, 'fsid': 'eb7f03c3-9691-4766-8999-6f4f526cda71', 'created': '2024-09-13T09:25:30.379173+0000', 'modified': '2024-09-13T09:27:59.517478+0000', 'last_up_change': '2024-09-13T09:27:01.686116+0000', 'last_in_change': '2024-09-13T09:27:33.261291+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:25:41.357433+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:25:42.578616+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:25:51.228779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 125, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '54ea2bd8-55b0-4ded-8a16-041261c5ba5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6816', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6817', 'nonce': 2906107514}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6818', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6819', 'nonce': 2906107514}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6822', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6823', 'nonce': 2906107514}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6820', 'nonce': 2906107514}, {'type': 'v1', 'addr': '172.21.15.63:6821', 'nonce': 2906107514}]}, 'public_addr': '172.21.15.63:6817/2906107514', 'cluster_addr': '172.21.15.63:6819/2906107514', 'heartbeat_back_addr': '172.21.15.63:6823/2906107514', 'heartbeat_front_addr': '172.21.15.63:6821/2906107514', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '0c95aee9-f871-4f6f-bbbd-1206db5d331c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6824', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6825', 'nonce': 3622854518}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6826', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6827', 'nonce': 3622854518}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6830', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6831', 'nonce': 3622854518}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6828', 'nonce': 3622854518}, {'type': 'v1', 'addr': '172.21.15.63:6829', 'nonce': 3622854518}]}, 'public_addr': '172.21.15.63:6825/3622854518', 'cluster_addr': '172.21.15.63:6827/3622854518', 'heartbeat_back_addr': '172.21.15.63:6831/3622854518', 'heartbeat_front_addr': '172.21.15.63:6829/3622854518', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f8253db2-d4da-46b5-95cf-cd5c9ebd7328', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6800', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6801', 'nonce': 2105659441}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6802', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6803', 'nonce': 2105659441}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6806', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6807', 'nonce': 2105659441}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6804', 'nonce': 2105659441}, {'type': 'v1', 'addr': '172.21.15.63:6805', 'nonce': 2105659441}]}, 'public_addr': '172.21.15.63:6801/2105659441', 'cluster_addr': '172.21.15.63:6803/2105659441', 'heartbeat_back_addr': '172.21.15.63:6807/2105659441', 'heartbeat_front_addr': '172.21.15.63:6805/2105659441', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '67c11a52-baeb-4ea8-89ea-26018ffdc35a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 46, 'up_from': 74, 'up_thru': 77, 'down_at': 48, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6808', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6809', 'nonce': 4293004354}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6810', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6811', 'nonce': 4293004354}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6814', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6815', 'nonce': 4293004354}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6812', 'nonce': 4293004354}, {'type': 'v1', 'addr': '172.21.15.63:6813', 'nonce': 4293004354}]}, 'public_addr': '172.21.15.63:6809/4293004354', 'cluster_addr': '172.21.15.63:6811/4293004354', 'heartbeat_back_addr': '172.21.15.63:6815/4293004354', 'heartbeat_front_addr': '172.21.15.63:6813/4293004354', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '1cec9702-fa09-4d7c-b111-7d10b1950cfe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 1789187713}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 1789187713}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 1789187713}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 1789187713}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 1789187713}]}, 'public_addr': '172.21.15.99:6801/1789187713', 'cluster_addr': '172.21.15.99:6803/1789187713', 'heartbeat_back_addr': '172.21.15.99:6807/1789187713', 'heartbeat_front_addr': '172.21.15.99:6805/1789187713', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7c0b908a-65c9-426c-867c-cc8ed65084b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6809', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 3410135021}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6813', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 3410135021}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6821', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3410135021}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6817', 'nonce': 3410135021}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3410135021}]}, 'public_addr': '172.21.15.99:6811/3410135021', 'cluster_addr': '172.21.15.99:6815/3410135021', 'heartbeat_back_addr': '172.21.15.99:6823/3410135021', 'heartbeat_front_addr': '172.21.15.99:6819/3410135021', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '78d5990a-4535-4110-ba04-88f01ab9fe28', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6810', 'nonce': 4152410871}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6814', 'nonce': 4152410871}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6822', 'nonce': 4152410871}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 4152410871}, {'type': 'v1', 'addr': '172.21.15.99:6818', 'nonce': 4152410871}]}, 'public_addr': '172.21.15.99:6810/4152410871', 'cluster_addr': '172.21.15.99:6814/4152410871', 'heartbeat_back_addr': '172.21.15.99:6822/4152410871', 'heartbeat_front_addr': '172.21.15.99:6818/4152410871', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '53f8511c-729f-4423-8eac-813c01f8f72b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 109, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2923799929}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2923799929}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2923799929}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2923799929}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2923799929}]}, 'public_addr': '172.21.15.99:6825/2923799929', 'cluster_addr': '172.21.15.99:6827/2923799929', 'heartbeat_back_addr': '172.21.15.99:6831/2923799929', 'heartbeat_front_addr': '172.21.15.99:6829/2923799929', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.267895+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.161313+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.447964+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:38.021320+0000', 'dead_epoch': 48}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.883961+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.938146+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.627237+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:25:37.887523+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:27:59.887 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:27:59.888 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:27:59.888 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:27:59.882+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:27:59.951 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-13T09:27:59.968 INFO:teuthology.orchestra.run.smithi063.stdout:5746 write 0x7f40450 thru 0x7f4e3c7 (0xdf78 bytes) 2024-09-13T09:27:59.986 INFO:teuthology.orchestra.run.smithi063.stdout:5747 punch from 0x673ddb1 to 0x673e00a, (0x259 bytes) 2024-09-13T09:27:59.986 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:59.986 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:27:59.986 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:27:59.982+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:27:59.989 INFO:teuthology.orchestra.run.smithi063.stdout:5750 trunc from 0xa1cb0f6 to 0xc3f565a 2024-09-13T09:28:00.002 INFO:teuthology.orchestra.run.smithi063.stdout:5752 write 0x710186d thru 0x7110827 (0xefbb bytes) 2024-09-13T09:28:00.010 INFO:teuthology.orchestra.run.smithi063.stdout:5754 punch from 0x1479588 to 0x147e3a2, (0x4e1a bytes) 2024-09-13T09:28:00.017 INFO:teuthology.orchestra.run.smithi063.stdout:5755 trunc from 0xc3f565a to 0x850f57a 2024-09-13T09:28:00.038 INFO:teuthology.orchestra.run.smithi063.stdout:5756 read 0x5eec03d thru 0x5eee65a (0x261e bytes) 2024-09-13T09:28:00.039 INFO:teuthology.orchestra.run.smithi063.stdout:5757 writesame 0x183109a thru 0x1839f59 data_size 0x20(0x8ec0 bytes) 2024-09-13T09:28:00.048 INFO:teuthology.orchestra.run.smithi063.stdout:5758 read 0x6ce9457 thru 0x6ced537 (0x40e1 bytes) 2024-09-13T09:28:00.051 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.051 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.058 INFO:teuthology.orchestra.run.smithi063.stdout:5760 write 0x85fcee0 thru 0x86084be (0xb5df bytes) 2024-09-13T09:28:00.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:00.064 INFO:teuthology.orchestra.run.smithi063.stdout:5761 read 0x76de6a2 thru 0x76e186f (0x31ce bytes) 2024-09-13T09:28:00.064 INFO:teuthology.orchestra.run.smithi063.stdout:5762 read 0x1dcd6ea thru 0x1dda007 (0xc91e bytes) 2024-09-13T09:28:00.087 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:00.082+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:00.087 INFO:teuthology.orchestra.run.smithi063.stdout:5763 writesame 0xedc94b0 thru 0xedd0e4d data_size 0x3(0x799e bytes) 2024-09-13T09:28:00.107 INFO:teuthology.orchestra.run.smithi063.stdout:5764 trunc from 0xedd0e4e to 0x25d7788 2024-09-13T09:28:00.136 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:00.136 INFO:teuthology.orchestra.run.smithi063.stdout:5766 writesame 0x10263c4 thru 0x1032ee5 data_size 0x36(0xcb22 bytes) 2024-09-13T09:28:00.174 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.174 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.176 INFO:teuthology.orchestra.run.smithi063.stdout:5767 read 0x17525ff thru 0x1759572 (0x6f74 bytes) 2024-09-13T09:28:00.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:00.229 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:00.226+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:00.231 INFO:teuthology.orchestra.run.smithi063.stdout:5769 write 0x83b1731 thru 0x83bf2b1 (0xdb81 bytes) 2024-09-13T09:28:00.236 INFO:teuthology.orchestra.run.smithi063.stdout:5773 punch from 0x10bac88 to 0x10c9753, (0xeacb bytes) 2024-09-13T09:28:00.236 INFO:teuthology.orchestra.run.smithi063.stdout:5774 punch from 0x38d1441 to 0x38df81b, (0xe3da bytes) 2024-09-13T09:28:00.236 INFO:teuthology.orchestra.run.smithi063.stdout:5775 punch from 0x6c312d to 0x6c8e76, (0x5d49 bytes) 2024-09-13T09:28:00.237 INFO:teuthology.orchestra.run.smithi063.stdout:5776 write 0x1fa1846 thru 0x1fb1648 (0xfe03 bytes) 2024-09-13T09:28:00.239 INFO:teuthology.orchestra.run.smithi063.stdout:5777 punch from 0x2a1dc7e to 0x2a2b433, (0xd7b5 bytes) 2024-09-13T09:28:00.240 INFO:teuthology.orchestra.run.smithi063.stdout:5778 write 0x41a05b5 thru 0x41a78e2 (0x732e bytes) 2024-09-13T09:28:00.247 INFO:teuthology.orchestra.run.smithi063.stdout:5781 read 0x78cd45b thru 0x78d7a3c (0xa5e2 bytes) 2024-09-13T09:28:00.247 INFO:teuthology.orchestra.run.smithi063.stdout:5782 write 0x1346879 thru 0x1352958 (0xc0e0 bytes) 2024-09-13T09:28:00.253 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:28:00.256 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:28:00.269 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-13T09:27:59.526671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1535850345,"num_objects":1776,"num_object_clones":342,"num_object_copies":3552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1776,"num_whiteouts":264,"num_read":212971,"num_read_kb":660078,"num_write":27130,"num_write_kb":399473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3084,"num_bytes_recovered":211673521,"num_keys_recovered":658,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20134,"ondisk_log_size":20134,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":470940,"kb_used_data":91332,"kb_used_omap":25,"kb_used_meta":379366,"kb_avail":754503780,"statfs":{"total":773094113280,"available":772611870720,"internally_reserved":0,"allocated":93523968,"data_stored":479863385,"data_compressed":40519005,"data_compressed_allocated":45871104,"data_compressed_original":442182383,"omap_allocated":25946,"internal_metadata":388471462},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,28],"upper_bound":4},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11397597,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":13616,"num_read_kb":12079,"num_write":5171,"num_write_kb":9678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.047760"},"pg_stats":[{"pgid":"3.1f","version":"121'430","reported_seq":1728,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.403356+0000","last_change":"2024-09-13T09:26:39.600998+0000","last_active":"2024-09-13T09:27:56.403356+0000","last_peered":"2024-09-13T09:27:56.403356+0000","last_clean":"2024-09-13T09:27:56.403356+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:27:56.403356+0000","last_undegraded":"2024-09-13T09:27:56.403356+0000","last_fullsized":"2024-09-13T09:27:56.403356+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023223459999999999,"stat_sum":{"num_bytes":41177473,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":11,"num_read":2987,"num_read_kb":15929,"num_write":464,"num_write_kb":7559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"121'679","reported_seq":5271,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.486612+0000","last_change":"2024-09-13T09:27:05.744365+0000","last_active":"2024-09-13T09:27:55.486612+0000","last_peered":"2024-09-13T09:27:55.486612+0000","last_clean":"2024-09-13T09:27:55.486612+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:27:55.486612+0000","last_undegraded":"2024-09-13T09:27:55.486612+0000","last_fullsized":"2024-09-13T09:27:55.486612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015028039999999999,"stat_sum":{"num_bytes":34456411,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":4,"num_read":6494,"num_read_kb":13244,"num_write":1002,"num_write_kb":6773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"122'460","reported_seq":4057,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.558900+0000","last_change":"2024-09-13T09:27:39.353427+0000","last_active":"2024-09-13T09:27:56.558900+0000","last_peered":"2024-09-13T09:27:56.558900+0000","last_clean":"2024-09-13T09:27:56.558900+0000","last_became_active":"2024-09-13T09:27:39.352330+0000","last_became_peered":"2024-09-13T09:27:39.352330+0000","last_unstale":"2024-09-13T09:27:56.558900+0000","last_undegraded":"2024-09-13T09:27:56.558900+0000","last_fullsized":"2024-09-13T09:27:56.558900+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:56.346789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016313740000000001,"stat_sum":{"num_bytes":57790951,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":9,"num_read":4071,"num_read_kb":20268,"num_write":541,"num_write_kb":11276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":12545300,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"122'506","reported_seq":5119,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.561886+0000","last_change":"2024-09-13T09:27:38.348001+0000","last_active":"2024-09-13T09:27:56.561886+0000","last_peered":"2024-09-13T09:27:56.561886+0000","last_clean":"2024-09-13T09:27:56.561886+0000","last_became_active":"2024-09-13T09:27:38.346735+0000","last_became_peered":"2024-09-13T09:27:38.346735+0000","last_unstale":"2024-09-13T09:27:56.561886+0000","last_undegraded":"2024-09-13T09:27:56.561886+0000","last_fullsized":"2024-09-13T09:27:56.561886+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:28:23.094925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001768452,"stat_sum":{"num_bytes":67602354,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":8,"num_read":4956,"num_read_kb":17791,"num_write":644,"num_write_kb":13016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":15156139,"num_keys_recovered":138,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"122'480","reported_seq":2899,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501631+0000","last_change":"2024-09-13T09:27:40.167421+0000","last_active":"2024-09-13T09:27:56.501631+0000","last_peered":"2024-09-13T09:27:56.501631+0000","last_clean":"2024-09-13T09:27:56.501631+0000","last_became_active":"2024-09-13T09:27:40.165702+0000","last_became_peered":"2024-09-13T09:27:40.165702+0000","last_unstale":"2024-09-13T09:27:56.501631+0000","last_undegraded":"2024-09-13T09:27:56.501631+0000","last_fullsized":"2024-09-13T09:27:56.501631+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:21:08.885741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201516,"num_objects":51,"num_object_clones":13,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":12,"num_read":3733,"num_read_kb":10627,"num_write":552,"num_write_kb":6214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":9930624,"num_keys_recovered":72,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"121'550","reported_seq":11532,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.568142+0000","last_change":"2024-09-13T09:27:37.336790+0000","last_active":"2024-09-13T09:27:56.568142+0000","last_peered":"2024-09-13T09:27:56.568142+0000","last_clean":"2024-09-13T09:27:56.568142+0000","last_became_active":"2024-09-13T09:27:37.334754+0000","last_became_peered":"2024-09-13T09:27:37.334754+0000","last_unstale":"2024-09-13T09:27:56.568142+0000","last_undegraded":"2024-09-13T09:27:56.568142+0000","last_fullsized":"2024-09-13T09:27:56.568142+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:35:54.865010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083156500000000004,"stat_sum":{"num_bytes":46971248,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":6,"num_read":12925,"num_read_kb":21496,"num_write":634,"num_write_kb":6617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":8205452,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"121'507","reported_seq":2623,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.415353+0000","last_change":"2024-09-13T09:27:05.741503+0000","last_active":"2024-09-13T09:27:56.415353+0000","last_peered":"2024-09-13T09:27:56.415353+0000","last_clean":"2024-09-13T09:27:56.415353+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:27:56.415353+0000","last_undegraded":"2024-09-13T09:27:56.415353+0000","last_fullsized":"2024-09-13T09:27:56.415353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059563900000000002,"stat_sum":{"num_bytes":29443717,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":7,"num_read":3351,"num_read_kb":9986,"num_write":535,"num_write_kb":8021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"121'490","reported_seq":1666,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.614985+0000","last_change":"2024-09-13T09:27:03.781188+0000","last_active":"2024-09-13T09:27:55.614985+0000","last_peered":"2024-09-13T09:27:55.614985+0000","last_clean":"2024-09-13T09:27:55.614985+0000","last_became_active":"2024-09-13T09:27:03.711948+0000","last_became_peered":"2024-09-13T09:27:03.711948+0000","last_unstale":"2024-09-13T09:27:55.614985+0000","last_undegraded":"2024-09-13T09:27:55.614985+0000","last_fullsized":"2024-09-13T09:27:55.614985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:36:42.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013302850000000001,"stat_sum":{"num_bytes":29701076,"num_objects":53,"num_object_clones":7,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":6,"num_read":3304,"num_read_kb":15167,"num_write":512,"num_write_kb":13675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":8502479,"num_keys_recovered":39,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"121'609","reported_seq":1524,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.601987+0000","last_change":"2024-09-13T09:26:42.575210+0000","last_active":"2024-09-13T09:27:55.601987+0000","last_peered":"2024-09-13T09:27:55.601987+0000","last_clean":"2024-09-13T09:27:55.601987+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:27:55.601987+0000","last_undegraded":"2024-09-13T09:27:55.601987+0000","last_fullsized":"2024-09-13T09:27:55.601987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:47.404587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10535024,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":5,"num_read":3079,"num_read_kb":4895,"num_write":723,"num_write_kb":3524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"122'441","reported_seq":5705,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.522068+0000","last_change":"2024-09-13T09:25:55.201367+0000","last_active":"2024-09-13T09:27:56.522068+0000","last_peered":"2024-09-13T09:27:56.522068+0000","last_clean":"2024-09-13T09:27:56.522068+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:27:56.522068+0000","last_undegraded":"2024-09-13T09:27:56.522068+0000","last_fullsized":"2024-09-13T09:27:56.522068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020104630000000001,"stat_sum":{"num_bytes":53405795,"num_objects":59,"num_object_clones":16,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":8,"num_read":5562,"num_read_kb":29365,"num_write":556,"num_write_kb":16204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"121'387","reported_seq":7499,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417871+0000","last_change":"2024-09-13T09:27:05.750137+0000","last_active":"2024-09-13T09:27:56.417871+0000","last_peered":"2024-09-13T09:27:56.417871+0000","last_clean":"2024-09-13T09:27:56.417871+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:27:56.417871+0000","last_undegraded":"2024-09-13T09:27:56.417871+0000","last_fullsized":"2024-09-13T09:27:56.417871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066978700000000003,"stat_sum":{"num_bytes":26932203,"num_objects":51,"num_object_clones":7,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":6,"num_read":7428,"num_read_kb":26689,"num_write":415,"num_write_kb":8419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"121'429","reported_seq":5544,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.087395+0000","last_change":"2024-09-13T09:27:37.337476+0000","last_active":"2024-09-13T09:27:56.087395+0000","last_peered":"2024-09-13T09:27:56.087395+0000","last_clean":"2024-09-13T09:27:56.087395+0000","last_became_active":"2024-09-13T09:27:37.335199+0000","last_became_peered":"2024-09-13T09:27:37.335199+0000","last_unstale":"2024-09-13T09:27:56.087395+0000","last_undegraded":"2024-09-13T09:27:56.087395+0000","last_fullsized":"2024-09-13T09:27:56.087395+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:24:06.331221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011424880000000001,"stat_sum":{"num_bytes":46596591,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":8,"num_read":5711,"num_read_kb":12853,"num_write":460,"num_write_kb":5527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":4959398,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":132,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501625+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:27:56.501625+0000","last_peered":"2024-09-13T09:27:56.501625+0000","last_clean":"2024-09-13T09:27:56.501625+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:27:56.501625+0000","last_undegraded":"2024-09-13T09:27:56.501625+0000","last_fullsized":"2024-09-13T09:27:56.501625+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"121'420","reported_seq":1244,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.415720+0000","last_change":"2024-09-13T09:27:05.753951+0000","last_active":"2024-09-13T09:27:56.415720+0000","last_peered":"2024-09-13T09:27:56.415720+0000","last_clean":"2024-09-13T09:27:56.415720+0000","last_became_active":"2024-09-13T09:27:05.753131+0000","last_became_peered":"2024-09-13T09:27:05.753131+0000","last_unstale":"2024-09-13T09:27:56.415720+0000","last_undegraded":"2024-09-13T09:27:56.415720+0000","last_fullsized":"2024-09-13T09:27:56.415720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:21.776084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019671020000000001,"stat_sum":{"num_bytes":69325118,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":8,"num_read":1004,"num_read_kb":6232,"num_write":503,"num_write_kb":8017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":140,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.501774+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:27:56.501774+0000","last_peered":"2024-09-13T09:27:56.501774+0000","last_clean":"2024-09-13T09:27:56.501774+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:27:56.501774+0000","last_undegraded":"2024-09-13T09:27:56.501774+0000","last_fullsized":"2024-09-13T09:27:56.501774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"121'710","reported_seq":3200,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.395622+0000","last_change":"2024-09-13T09:25:55.184890+0000","last_active":"2024-09-13T09:27:56.395622+0000","last_peered":"2024-09-13T09:27:56.395622+0000","last_clean":"2024-09-13T09:27:56.395622+0000","last_became_active":"2024-09-13T09:25:53.642512+0000","last_became_peered":"2024-09-13T09:25:53.642512+0000","last_unstale":"2024-09-13T09:27:56.395622+0000","last_undegraded":"2024-09-13T09:27:56.395622+0000","last_fullsized":"2024-09-13T09:27:56.395622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:40.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067051199999999995,"stat_sum":{"num_bytes":35108989,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":6,"num_read":5051,"num_read_kb":18443,"num_write":993,"num_write_kb":21566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.481556+0000","last_change":"2024-09-13T09:27:34.289812+0000","last_active":"2024-09-13T09:27:55.481556+0000","last_peered":"2024-09-13T09:27:55.481556+0000","last_clean":"2024-09-13T09:27:55.481556+0000","last_became_active":"2024-09-13T09:27:34.289466+0000","last_became_peered":"2024-09-13T09:27:34.289466+0000","last_unstale":"2024-09-13T09:27:55.481556+0000","last_undegraded":"2024-09-13T09:27:55.481556+0000","last_fullsized":"2024-09-13T09:27:55.481556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:42.497510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"121'652","reported_seq":5586,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.486868+0000","last_change":"2024-09-13T09:27:05.741382+0000","last_active":"2024-09-13T09:27:55.486868+0000","last_peered":"2024-09-13T09:27:55.486868+0000","last_clean":"2024-09-13T09:27:55.486868+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:27:55.486868+0000","last_undegraded":"2024-09-13T09:27:55.486868+0000","last_fullsized":"2024-09-13T09:27:55.486868+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019164130000000001,"stat_sum":{"num_bytes":74457018,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":9,"num_read":7238,"num_read_kb":25922,"num_write":963,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:54.472800+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:27:54.472800+0000","last_peered":"2024-09-13T09:27:54.472800+0000","last_clean":"2024-09-13T09:27:54.472800+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:27:54.472800+0000","last_undegraded":"2024-09-13T09:27:54.472800+0000","last_fullsized":"2024-09-13T09:27:54.472800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"121'689","reported_seq":5426,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.411416+0000","last_change":"2024-09-13T09:26:42.570158+0000","last_active":"2024-09-13T09:27:56.411416+0000","last_peered":"2024-09-13T09:27:56.411416+0000","last_clean":"2024-09-13T09:27:56.411416+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:27:56.411416+0000","last_undegraded":"2024-09-13T09:27:56.411416+0000","last_fullsized":"2024-09-13T09:27:56.411416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53785345,"num_objects":66,"num_object_clones":12,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":10,"num_read":7565,"num_read_kb":24285,"num_write":904,"num_write_kb":8412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"121'1254","reported_seq":2685,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417251+0000","last_change":"2024-09-13T09:27:40.165078+0000","last_active":"2024-09-13T09:27:56.417251+0000","last_peered":"2024-09-13T09:27:56.417251+0000","last_clean":"2024-09-13T09:27:56.417251+0000","last_became_active":"2024-09-13T09:27:40.164156+0000","last_became_peered":"2024-09-13T09:27:40.164156+0000","last_unstale":"2024-09-13T09:27:56.417251+0000","last_undegraded":"2024-09-13T09:27:56.417251+0000","last_fullsized":"2024-09-13T09:27:56.417251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:59:58.977950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13597679,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":6,"num_read":6916,"num_read_kb":15309,"num_write":1974,"num_write_kb":6812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":6885568,"num_keys_recovered":105,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":263,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490814+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:27:55.490814+0000","last_peered":"2024-09-13T09:27:55.490814+0000","last_clean":"2024-09-13T09:27:55.490814+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:27:55.490814+0000","last_undegraded":"2024-09-13T09:27:55.490814+0000","last_fullsized":"2024-09-13T09:27:55.490814+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":119,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:54.482791+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:27:54.482791+0000","last_peered":"2024-09-13T09:27:54.482791+0000","last_clean":"2024-09-13T09:27:54.482791+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:27:54.482791+0000","last_undegraded":"2024-09-13T09:27:54.482791+0000","last_fullsized":"2024-09-13T09:27:54.482791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"121'889","reported_seq":8398,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.080626+0000","last_change":"2024-09-13T09:25:55.183972+0000","last_active":"2024-09-13T09:27:56.080626+0000","last_peered":"2024-09-13T09:27:56.080626+0000","last_clean":"2024-09-13T09:27:56.080626+0000","last_became_active":"2024-09-13T09:25:53.642483+0000","last_became_peered":"2024-09-13T09:25:53.642483+0000","last_unstale":"2024-09-13T09:27:56.080626+0000","last_undegraded":"2024-09-13T09:27:56.080626+0000","last_fullsized":"2024-09-13T09:27:56.080626+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:03:57.875908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054925099999999995,"stat_sum":{"num_bytes":64588489,"num_objects":58,"num_object_clones":10,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":9,"num_read":10690,"num_read_kb":23133,"num_write":1323,"num_write_kb":18418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":145,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490844+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:27:55.490844+0000","last_peered":"2024-09-13T09:27:55.490844+0000","last_clean":"2024-09-13T09:27:55.490844+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:27:55.490844+0000","last_undegraded":"2024-09-13T09:27:55.490844+0000","last_fullsized":"2024-09-13T09:27:55.490844+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"121'539","reported_seq":10201,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417876+0000","last_change":"2024-09-13T09:27:38.350122+0000","last_active":"2024-09-13T09:27:56.417876+0000","last_peered":"2024-09-13T09:27:56.417876+0000","last_clean":"2024-09-13T09:27:56.417876+0000","last_became_active":"2024-09-13T09:27:38.348807+0000","last_became_peered":"2024-09-13T09:27:38.348807+0000","last_unstale":"2024-09-13T09:27:56.417876+0000","last_undegraded":"2024-09-13T09:27:56.417876+0000","last_fullsized":"2024-09-13T09:27:56.417876+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:08.088219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001152251,"stat_sum":{"num_bytes":86966958,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":11,"num_read":9772,"num_read_kb":38350,"num_write":673,"num_write_kb":31494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":37856068,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":136,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.497662+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:27:55.497662+0000","last_peered":"2024-09-13T09:27:55.497662+0000","last_clean":"2024-09-13T09:27:55.497662+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:27:55.497662+0000","last_undegraded":"2024-09-13T09:27:55.497662+0000","last_fullsized":"2024-09-13T09:27:55.497662+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"121'1861","reported_seq":6509,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.694854+0000","last_change":"2024-09-13T09:27:05.745925+0000","last_active":"2024-09-13T09:27:55.694854+0000","last_peered":"2024-09-13T09:27:55.694854+0000","last_clean":"2024-09-13T09:27:55.694854+0000","last_became_active":"2024-09-13T09:27:05.744803+0000","last_became_peered":"2024-09-13T09:27:05.744803+0000","last_unstale":"2024-09-13T09:27:55.694854+0000","last_undegraded":"2024-09-13T09:27:55.694854+0000","last_fullsized":"2024-09-13T09:27:55.694854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1861,"log_dups_size":0,"ondisk_log_size":1861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:40.185419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020918820000000002,"stat_sum":{"num_bytes":25584455,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":5,"num_read":13735,"num_read_kb":16194,"num_write":3351,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":135,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.490772+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:27:55.490772+0000","last_peered":"2024-09-13T09:27:55.490772+0000","last_clean":"2024-09-13T09:27:55.490772+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:27:55.490772+0000","last_undegraded":"2024-09-13T09:27:55.490772+0000","last_fullsized":"2024-09-13T09:27:55.490772+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"120'486","reported_seq":1871,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.460423+0000","last_change":"2024-09-13T09:27:05.745269+0000","last_active":"2024-09-13T09:27:55.460423+0000","last_peered":"2024-09-13T09:27:55.460423+0000","last_clean":"2024-09-13T09:27:55.460423+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:27:55.460423+0000","last_undegraded":"2024-09-13T09:27:55.460423+0000","last_fullsized":"2024-09-13T09:27:55.460423+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014663,"stat_sum":{"num_bytes":23606372,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":9,"num_read":2940,"num_read_kb":13676,"num_write":552,"num_write_kb":6739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"121'398","reported_seq":11067,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417645+0000","last_change":"2024-09-13T09:27:39.357803+0000","last_active":"2024-09-13T09:27:56.417645+0000","last_peered":"2024-09-13T09:27:56.417645+0000","last_clean":"2024-09-13T09:27:56.417645+0000","last_became_active":"2024-09-13T09:27:39.355670+0000","last_became_peered":"2024-09-13T09:27:39.355670+0000","last_unstale":"2024-09-13T09:27:56.417645+0000","last_undegraded":"2024-09-13T09:27:56.417645+0000","last_fullsized":"2024-09-13T09:27:56.417645+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:18.749072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091493900000000003,"stat_sum":{"num_bytes":55902485,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":10,"num_read":10451,"num_read_kb":21041,"num_write":430,"num_write_kb":14364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":9133614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"121'382","reported_seq":3629,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.112269+0000","last_change":"2024-09-13T09:27:39.354051+0000","last_active":"2024-09-13T09:27:56.112269+0000","last_peered":"2024-09-13T09:27:56.112269+0000","last_clean":"2024-09-13T09:27:56.112269+0000","last_became_active":"2024-09-13T09:27:39.352709+0000","last_became_peered":"2024-09-13T09:27:39.352709+0000","last_unstale":"2024-09-13T09:27:56.112269+0000","last_undegraded":"2024-09-13T09:27:56.112269+0000","last_fullsized":"2024-09-13T09:27:56.112269+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:53:01.998737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083340200000000001,"stat_sum":{"num_bytes":38529235,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":9,"num_read":3008,"num_read_kb":20333,"num_write":391,"num_write_kb":15728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":5132540,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"121'512","reported_seq":4076,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:55.633966+0000","last_change":"2024-09-13T09:27:05.743332+0000","last_active":"2024-09-13T09:27:55.633966+0000","last_peered":"2024-09-13T09:27:55.633966+0000","last_clean":"2024-09-13T09:27:55.633966+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:27:55.633966+0000","last_undegraded":"2024-09-13T09:27:55.633966+0000","last_fullsized":"2024-09-13T09:27:55.633966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020774420000000001,"stat_sum":{"num_bytes":68568851,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":10,"num_read":3792,"num_read_kb":27296,"num_write":599,"num_write_kb":19857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"121'527","reported_seq":7283,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.418174+0000","last_change":"2024-09-13T09:27:05.746491+0000","last_active":"2024-09-13T09:27:56.418174+0000","last_peered":"2024-09-13T09:27:56.418174+0000","last_clean":"2024-09-13T09:27:56.418174+0000","last_became_active":"2024-09-13T09:27:05.745595+0000","last_became_peered":"2024-09-13T09:27:05.745595+0000","last_unstale":"2024-09-13T09:27:56.418174+0000","last_undegraded":"2024-09-13T09:27:56.418174+0000","last_fullsized":"2024-09-13T09:27:56.418174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:38.773883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056797900000000003,"stat_sum":{"num_bytes":42585420,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":4,"num_read":7699,"num_read_kb":21253,"num_write":745,"num_write_kb":15259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"121'398","reported_seq":14221,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.401322+0000","last_change":"2024-09-13T09:25:55.220014+0000","last_active":"2024-09-13T09:27:56.401322+0000","last_peered":"2024-09-13T09:27:56.401322+0000","last_clean":"2024-09-13T09:27:56.401322+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:27:56.401322+0000","last_undegraded":"2024-09-13T09:27:56.401322+0000","last_fullsized":"2024-09-13T09:27:56.401322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015667859999999999,"stat_sum":{"num_bytes":82298447,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":14,"num_read":13865,"num_read_kb":35794,"num_write":479,"num_write_kb":29504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"121'1075","reported_seq":8014,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417703+0000","last_change":"2024-09-13T09:27:40.166348+0000","last_active":"2024-09-13T09:27:56.417703+0000","last_peered":"2024-09-13T09:27:56.417703+0000","last_clean":"2024-09-13T09:27:56.417703+0000","last_became_active":"2024-09-13T09:27:40.164774+0000","last_became_peered":"2024-09-13T09:27:40.164774+0000","last_unstale":"2024-09-13T09:27:56.417703+0000","last_undegraded":"2024-09-13T09:27:56.417703+0000","last_fullsized":"2024-09-13T09:27:56.417703+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:43.498002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021934559999999999,"stat_sum":{"num_bytes":77899202,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":12,"num_read":10748,"num_read_kb":23563,"num_write":1746,"num_write_kb":18987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":13609565,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"121'411","reported_seq":6266,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.202806+0000","last_change":"2024-09-13T09:27:37.336194+0000","last_active":"2024-09-13T09:27:56.202806+0000","last_peered":"2024-09-13T09:27:56.202806+0000","last_clean":"2024-09-13T09:27:56.202806+0000","last_became_active":"2024-09-13T09:27:37.335370+0000","last_became_peered":"2024-09-13T09:27:37.335370+0000","last_unstale":"2024-09-13T09:27:56.202806+0000","last_undegraded":"2024-09-13T09:27:56.202806+0000","last_fullsized":"2024-09-13T09:27:56.202806+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:17:20.967570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001554986,"stat_sum":{"num_bytes":59679891,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":6,"num_read":5598,"num_read_kb":29560,"num_write":462,"num_write_kb":13748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":13410696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"121'761","reported_seq":3605,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417618+0000","last_change":"2024-09-13T09:26:42.574072+0000","last_active":"2024-09-13T09:27:56.417618+0000","last_peered":"2024-09-13T09:27:56.417618+0000","last_clean":"2024-09-13T09:27:56.417618+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:27:56.417618+0000","last_undegraded":"2024-09-13T09:27:56.417618+0000","last_fullsized":"2024-09-13T09:27:56.417618+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072136100000000001,"stat_sum":{"num_bytes":37693572,"num_objects":64,"num_object_clones":12,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":10,"num_read":4846,"num_read_kb":23323,"num_write":1104,"num_write_kb":9498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"121'705","reported_seq":7674,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.417713+0000","last_change":"2024-09-13T09:27:41.168314+0000","last_active":"2024-09-13T09:27:56.417713+0000","last_peered":"2024-09-13T09:27:56.417713+0000","last_clean":"2024-09-13T09:27:56.417713+0000","last_became_active":"2024-09-13T09:27:41.165699+0000","last_became_peered":"2024-09-13T09:27:41.165699+0000","last_unstale":"2024-09-13T09:27:56.417713+0000","last_undegraded":"2024-09-13T09:27:56.417713+0000","last_fullsized":"2024-09-13T09:27:56.417713+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:03.888324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020479679999999998,"stat_sum":{"num_bytes":49286710,"num_objects":56,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":9,"num_read":9644,"num_read_kb":25247,"num_write":885,"num_write_kb":8751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":9211408,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"121'439","reported_seq":5275,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.242228+0000","last_change":"2024-09-13T09:27:17.867086+0000","last_active":"2024-09-13T09:27:56.242228+0000","last_peered":"2024-09-13T09:27:56.242228+0000","last_clean":"2024-09-13T09:27:56.242228+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:27:56.242228+0000","last_undegraded":"2024-09-13T09:27:56.242228+0000","last_fullsized":"2024-09-13T09:27:56.242228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57888450,"num_objects":55,"num_object_clones":7,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":4,"num_read":5013,"num_read_kb":22896,"num_write":583,"num_write_kb":14850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"122'938","reported_seq":7127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-13T09:27:56.557018+0000","last_change":"2024-09-13T09:27:39.358188+0000","last_active":"2024-09-13T09:27:56.557018+0000","last_peered":"2024-09-13T09:27:56.557018+0000","last_clean":"2024-09-13T09:27:56.557018+0000","last_became_active":"2024-09-13T09:27:39.355967+0000","last_became_peered":"2024-09-13T09:27:39.355967+0000","last_unstale":"2024-09-13T09:27:56.557018+0000","last_undegraded":"2024-09-13T09:27:56.557018+0000","last_fullsized":"2024-09-13T09:27:56.557018+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:08.064206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001137522,"stat_sum":{"num_bytes":60092913,"num_objects":72,"num_object_clones":18,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":13,"num_read":9679,"num_read_kb":29818,"num_write":1271,"num_write_kb":11245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":10989136,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1535259958,"num_objects":1772,"num_object_clones":342,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":264,"num_read":212855,"num_read_kb":659978,"num_write":26969,"num_write_kb":397055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3082,"num_bytes_recovered":211673483,"num_keys_recovered":658,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81969152,"data_stored":472534405,"data_compressed":40415923,"data_compressed_allocated":45731840,"data_compressed_original":440234151,"omap_allocated":8553,"internal_metadata":0},"log_size":20014,"ondisk_log_size":20014,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70016,"kb_used_data":16760,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94301824,"statfs":{"total":96636764160,"available":96565067776,"internally_reserved":0,"allocated":17162240,"data_stored":93311690,"data_compressed":7952238,"data_compressed_allocated":8224768,"data_compressed_original":85614719,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.78300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.224,"5min":1.256,"15min":1.256},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.76900000000000002,"15min":0.75800000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85599999999999998,"15min":0.86099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.875,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.2649999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.099},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1910000000000001,"15min":1.2529999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.484,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.96199999999999997,"15min":0.97999999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.875},{"interface":"front","average":{"1min":0.84799999999999998,"5min":1.0029999999999999,"15min":1.0289999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.175,"5min":1.357,"15min":1.357},"last":1.1459999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":1.0409999999999999,"15min":1.0649999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.125},{"interface":"front","average":{"1min":0.89900000000000002,"5min":1.0029999999999999,"15min":1.0209999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.90000000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44316,"kb_used_data":4940,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327524,"statfs":{"total":96636764160,"available":96591384576,"internally_reserved":0,"allocated":5058560,"data_stored":18964197,"data_compressed":1904445,"data_compressed_allocated":2068480,"data_compressed_original":17003491,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.86899999999999999,"15min":0.89700000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.1100000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.69999999999999996},"min":{"1min":0.46800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0800000000000001,"5min":1.173,"15min":1.173},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.92500000000000004,"15min":0.96399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.161,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.81100000000000005,"15min":0.82199999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.1599999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.258}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1439999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.997,"15min":1.0289999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.161,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1970000000000001,"5min":5.149,"15min":5.149},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2490000000000001,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":1.387}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89600000000000002,"15min":0.90900000000000003},"min":{"1min":0.48899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.181,"5min":2.032,"15min":2.032},"last":1.3069999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.0980000000000001,"15min":1.119},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.2509999999999999,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":1.048}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.90800000000000003,"15min":0.90400000000000003},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2949999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":1.3280000000000001,"15min":1.3939999999999999},"min":{"1min":0.59699999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.234,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.4990000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84720,"kb_used_data":13416,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94287120,"statfs":{"total":96636764160,"available":96550010880,"internally_reserved":0,"allocated":13737984,"data_stored":68967529,"data_compressed":6019523,"data_compressed_allocated":6807552,"data_compressed_original":63318073,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.81499999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1160000000000001,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.496,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0920000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.081,"5min":1.083,"15min":1.083},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.85099999999999998,"15min":0.86699999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1539999999999999,"5min":1.242,"15min":1.242},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.503,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.393,"5min":1.554,"15min":1.554},"last":1.1879999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.94699999999999995,"15min":0.97099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3979999999999999,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.97499999999999998,"15min":0.98899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.617,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":0.99399999999999999},"min":{"1min":0.52400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.23},{"interface":"front","average":{"1min":0.996,"5min":1.091,"15min":1.107},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6000000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.873}]}]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63252,"kb_used_data":18124,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308588,"statfs":{"total":96636764160,"available":96571994112,"internally_reserved":0,"allocated":18558976,"data_stored":100313080,"data_compressed":9368840,"data_compressed_allocated":10174464,"data_compressed_original":93432607,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.82299999999999995},"min":{"1min":0.52300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.274,"5min":1.276,"15min":1.276},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.88500000000000001,"15min":0.89800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.143,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.34699999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.90300000000000002,"15min":0.92400000000000004},"min":{"1min":0.57899999999999996,"5min":0.434,"15min":0.434},"max":{"1min":1.2050000000000001,"5min":1.298,"15min":1.298},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.56000000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.234,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":1.2110000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.374}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.879,"15min":0.89300000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.443},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88,"15min":0.88800000000000001},"min":{"1min":0.45800000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.351,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.1799999999999999,"5min":1.304,"15min":1.304},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96499999999999997,"15min":0.97299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.381,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:27:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.038,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.452,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.97699999999999998,"15min":0.97999999999999998},"min":{"1min":0.63800000000000001,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.4139999999999999,"5min":1.605,"15min":1.605},"last":0.69899999999999995}]}]},{"osd":3,"up_from":74,"seq":317827579916,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80988,"kb_used_data":11988,"kb_used_omap":14,"kb_used_meta":68977,"kb_avail":94290852,"statfs":{"total":96636764160,"available":96553832448,"internally_reserved":0,"allocated":12275712,"data_stored":66270059,"data_compressed":4701353,"data_compressed_allocated":5734400,"data_compressed_original":60846168,"omap_allocated":14794,"internal_metadata":70633014},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38656,"kb_used_data":7544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94333184,"statfs":{"total":96636764160,"available":96597180416,"internally_reserved":0,"allocated":7725056,"data_stored":34817629,"data_compressed":2581559,"data_compressed_allocated":3407872,"data_compressed_original":31549033,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.624,"15min":0.63},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.089,"5min":1.484,"15min":1.484},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.66900000000000004,"15min":0.68100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89800000000000002,"5min":1.335,"15min":1.335},"last":0.51200000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.65400000000000003,"15min":0.66000000000000003},"min":{"1min":0.36299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.98899999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.2310000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.98199999999999998,"15min":1.0009999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3420000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.306,"5min":1.587,"15min":1.587},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0049999999999999,"15min":0.98299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":7.6879999999999997,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.028,"15min":1.036},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.488,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.1779999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":1.0820000000000001,"15min":1.1040000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.28,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.0349999999999999,"15min":1.048},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.3420000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0489999999999999,"15min":1.056},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5029999999999999,"5min":2.048,"15min":2.048},"last":1.129},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.1539999999999999,"15min":1.169},"min":{"1min":0.69399999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.556,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.89000000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44392,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94327448,"statfs":{"total":96636764160,"available":96591306752,"internally_reserved":0,"allocated":13594624,"data_stored":76297071,"data_compressed":6385127,"data_compressed_allocated":7409664,"data_compressed_original":71908720,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[26,28],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.61399999999999999,"15min":0.63300000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.78400000000000003,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":12.441000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.70299999999999996,"15min":0.72099999999999997},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.98499999999999999,"5min":1.333,"15min":1.333},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.014,"5min":1.153,"15min":1.153},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.85699999999999998,"15min":0.90300000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91400000000000003,"5min":5.9420000000000002,"15min":5.9420000000000002},"last":1.8540000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.555}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1950000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":2.7130000000000001},{"interface":"front","average":{"1min":3.133,"5min":1.4350000000000001,"15min":1.1519999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":49.905999999999999,"5min":49.905999999999999,"15min":49.905999999999999},"last":12.487}]},{"osd":5,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":1.01,"15min":1.034},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2170000000000001,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.95199999999999996,"15min":0.95799999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":8.0109999999999992}]},{"osd":6,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":1.0449999999999999,"15min":1.079},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.371,"5min":1.704,"15min":1.704},"last":12.519},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0669999999999999,"15min":1.083},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.258,"5min":1.514,"15min":1.514},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.081,"15min":1.109},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2490000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":14.143000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.099,"15min":1.1259999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.427,"5min":1.7,"15min":1.7},"last":1.891}]}]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44600,"kb_used_data":5284,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94327240,"statfs":{"total":96636764160,"available":96591093760,"internally_reserved":0,"allocated":5410816,"data_stored":20922130,"data_compressed":1605920,"data_compressed_allocated":2043904,"data_compressed_original":18509572,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.60699999999999998,"15min":0.61899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.74199999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.312},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.437,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.042,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.018,"5min":1.24,"15min":1.24},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.292,"5min":1.996,"15min":1.996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6379999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1910000000000001,"5min":1.214,"15min":1.214},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.004,"15min":1.012},"min":{"1min":0.623,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.522,"5min":1.651,"15min":1.651},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.96799999999999997,"15min":0.97599999999999998},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.26,"5min":1.448,"15min":1.448},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.96599999999999997,"15min":0.97399999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3300000000000001,"5min":1.333,"15min":1.333},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0569999999999999,"15min":1.073},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.357,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1,"15min":1.002},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12144640,"data_stored":75616188,"data_compressed":6385127,"data_compressed_allocated":7409664,"data_compressed_original":71908720,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6254592,"data_stored":33440486,"data_compressed":2542374,"data_compressed_allocated":3354624,"data_compressed_original":30843181,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3866624,"data_stored":20203385,"data_compressed":1605920,"data_compressed_allocated":2043904,"data_compressed_original":18509572,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10858496,"data_stored":65601401,"data_compressed":4701353,"data_compressed_allocated":5734400,"data_compressed_original":60846168,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17158144,"data_stored":99194844,"data_compressed":9347217,"data_compressed_allocated":10145792,"data_compressed_original":92964371,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12365824,"data_stored":68321133,"data_compressed":6019523,"data_compressed_allocated":6807552,"data_compressed_original":63318073,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3629056,"data_stored":18093249,"data_compressed":1895178,"data_compressed_allocated":2056192,"data_compressed_original":16802787,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15691776,"data_stored":92063719,"data_compressed":7919231,"data_compressed_allocated":8179712,"data_compressed_original":85041279,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:28:00.272 INFO:tasks.thrashosds.thrasher:Setting 3.18 to [2, 5] 2024-09-13T09:28:00.272 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.18', '2', '5'] 2024-09-13T09:28:00.273 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.18 2 5 2024-09-13T09:28:00.273 INFO:teuthology.orchestra.run.smithi063.stdout:5783 write 0xdc7bf6c thru 0xdc822d0 (0x6365 bytes) 2024-09-13T09:28:00.284 INFO:teuthology.orchestra.run.smithi063.stdout:5786 trunc from 0xdc822d1 to 0xd108612 2024-09-13T09:28:00.288 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:00.282+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:00.303 INFO:teuthology.orchestra.run.smithi063.stdout:5787 read 0x7926b55 thru 0x7935ea8 (0xf354 bytes) 2024-09-13T09:28:00.304 INFO:teuthology.orchestra.run.smithi063.stdout:5788 read 0x2160c57 thru 0x21692b8 (0x8662 bytes) 2024-09-13T09:28:00.305 INFO:teuthology.orchestra.run.smithi063.stdout:5790 writesame 0x6efa9b0 thru 0x6efaef7 data_size 0xd(0x548 bytes) 2024-09-13T09:28:00.312 INFO:teuthology.orchestra.run.smithi063.stdout:5792 read 0x20fd458 thru 0x20feaba (0x1663 bytes) 2024-09-13T09:28:00.322 INFO:teuthology.orchestra.run.smithi063.stdout:5793 trunc from 0xd108612 to 0x805b1a0 2024-09-13T09:28:00.337 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:00.337 INFO:teuthology.orchestra.run.smithi063.stdout:5795 read 0x16b9b23 thru 0x16bbca3 (0x2181 bytes) 2024-09-13T09:28:00.342 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.342 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.342 INFO:teuthology.orchestra.run.smithi063.stdout:5796 read 0x7763e82 thru 0x776b70c (0x788b bytes) 2024-09-13T09:28:00.353 INFO:teuthology.orchestra.run.smithi063.stdout:5798 trunc from 0x805b1a0 to 0x203050e 2024-09-13T09:28:00.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:00.389 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:00.386+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:00.421 INFO:teuthology.orchestra.run.smithi063.stdout:5801 trunc from 0x203050e to 0x140fd43 2024-09-13T09:28:00.445 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:00.446 INFO:teuthology.orchestra.run.smithi063.stdout:5803 read 0x54e2d6 thru 0x555fb7 (0x7ce2 bytes) 2024-09-13T09:28:00.463 INFO:teuthology.orchestra.run.smithi063.stdout:5804 write 0x36287c7 thru 0x363774a (0xef84 bytes) 2024-09-13T09:28:00.465 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.465 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.470 INFO:teuthology.orchestra.run.smithi063.stdout:5805 clone 44 order 25 su 1048576 sc 5 2024-09-13T09:28:00.475 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:00.518 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:00.514+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:00.590 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:00.584+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:00.629 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.630 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:00.692 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:00.684+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:00.717 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone43 from 0x363774b (overlap 0x140fd43) to 0x604bfa 2024-09-13T09:28:00.752 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.752 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:00.792 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:00.786+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:00.868 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.869 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:00.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:00.894 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:00.888+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:00.944 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-13T09:28:00.963 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:00.990 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:00.990 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:01.000 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:01.008 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:01.004+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:01.014 INFO:teuthology.orchestra.run.smithi063.stdout:5806 writesame 0xbdcfaf0 thru 0xbdd134f data_size 0x3(0x1860 bytes) 2024-09-13T09:28:01.015 INFO:teuthology.orchestra.run.smithi063.stdout:5807 write 0x6d2b0cb thru 0x6d30f40 (0x5e76 bytes) 2024-09-13T09:28:01.043 INFO:teuthology.orchestra.run.smithi063.stdout:5808 write 0xcb551cd thru 0xcb5982c (0x4660 bytes) 2024-09-13T09:28:01.050 INFO:teuthology.orchestra.run.smithi063.stdout:5810 read 0x5a6029a thru 0x5a6eb18 (0xe87f bytes) 2024-09-13T09:28:01.051 INFO:teuthology.orchestra.run.smithi063.stdout:5812 writesame 0x223665c thru 0x2241ea1 data_size 0x67(0xb846 bytes) 2024-09-13T09:28:01.072 INFO:teuthology.orchestra.run.smithi063.stdout:5813 write 0xeb8f35f thru 0xeb9dad5 (0xe777 bytes) 2024-09-13T09:28:01.079 INFO:teuthology.orchestra.run.smithi063.stdout:5814 read 0x592ad2a thru 0x59338d6 (0x8bad bytes) 2024-09-13T09:28:01.096 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:01.088+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:01.098 INFO:teuthology.orchestra.run.smithi063.stdout:5816 trunc from 0xeb9dad6 to 0x4d981ac 2024-09-13T09:28:01.133 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:01.133 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:01.134 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:01.144 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:01.144 INFO:teuthology.orchestra.run.smithi063.stdout:5817 write 0x2efa3e2 thru 0x2f081d8 (0xddf7 bytes) 2024-09-13T09:28:01.146 INFO:teuthology.orchestra.run.smithi063.stdout:5818 punch from 0x1b8eb4b to 0x1b9d78f, (0xec44 bytes) 2024-09-13T09:28:01.163 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:01.170 INFO:teuthology.orchestra.run.smithi063.stdout:5819 write 0xc6ed31c thru 0xc6f49f5 (0x76da bytes) 2024-09-13T09:28:01.177 INFO:teuthology.orchestra.run.smithi063.stdout:5821 punch from 0x5ecbc4a to 0x5ed9077, (0xd42d bytes) 2024-09-13T09:28:01.177 INFO:teuthology.orchestra.run.smithi063.stdout:5822 punch from 0xc3be128 to 0xc3c5c79, (0x7b51 bytes) 2024-09-13T09:28:01.178 INFO:teuthology.orchestra.run.smithi063.stdout:5823 write 0x150bcc5 thru 0x1515d25 (0xa061 bytes) 2024-09-13T09:28:01.181 INFO:teuthology.orchestra.run.smithi063.stdout:5826 write 0x84eeab1 thru 0x84f54cc (0x6a1c bytes) 2024-09-13T09:28:01.196 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:01.190+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:01.199 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:01.202 INFO:teuthology.orchestra.run.smithi063.stdout:5827 trunc from 0xc6f49f6 to 0x2ca9765 2024-09-13T09:28:01.230 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:01.245 INFO:teuthology.orchestra.run.smithi063.stdout:5828 trunc from 0x2ca9765 to 0xbeabbda 2024-09-13T09:28:01.257 INFO:teuthology.orchestra.run.smithi063.stdout:5829 write 0x66dc9fe thru 0x66e0cc6 (0x42c9 bytes) 2024-09-13T09:28:01.259 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:01.261 INFO:teuthology.orchestra.run.smithi063.stdout:5830 read 0xb0d3465 thru 0xb0e3319 (0xfeb5 bytes) 2024-09-13T09:28:01.261 INFO:teuthology.orchestra.run.smithi063.stdout:5831 write 0x38e243a thru 0x38e85c1 (0x6188 bytes) 2024-09-13T09:28:01.262 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:01.262 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:01.262 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:01.263 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:01.263 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:01.274 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:01.282 INFO:teuthology.orchestra.run.smithi063.stdout:5832 trunc from 0xbeabbda to 0x1fe703c 2024-09-13T09:28:01.293 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:01.296 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:01.292+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:01.299 INFO:teuthology.orchestra.run.smithi063.stdout:5833 read 0x1e46938 thru 0x1e5626b (0xf934 bytes) 2024-09-13T09:28:01.300 INFO:teuthology.orchestra.run.smithi063.stdout:5834 read 0x169c474 thru 0x169f81e (0x33ab bytes) 2024-09-13T09:28:01.300 INFO:teuthology.orchestra.run.smithi063.stdout:5835 read 0x1cae7a7 thru 0x1cbd96c (0xf1c6 bytes) 2024-09-13T09:28:01.301 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:01.301 INFO:teuthology.orchestra.run.smithi063.stdout:5838 read 0x1d8ca25 thru 0x1d993ee (0xc9ca bytes) 2024-09-13T09:28:01.302 INFO:teuthology.orchestra.run.smithi063.stdout:5839 write 0x1011fdc thru 0x1019d7a (0x7d9f bytes) 2024-09-13T09:28:01.305 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:01.315 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:01.316 INFO:teuthology.orchestra.run.smithi063.stdout:5842 write 0x2217569 thru 0x22208b6 (0x934e bytes) 2024-09-13T09:28:01.338 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:01.340 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:01.340 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:01.340 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:01.340 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:01.341 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:01.341 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:01.341 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:01.341 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:01.345 INFO:teuthology.orchestra.run.smithi063.stdout:5843 writesame 0x8a35f84 thru 0x8a41259 data_size 0xb(0xb2d6 bytes) 2024-09-13T09:28:01.348 INFO:teuthology.orchestra.run.smithi063.stdout:5844 punch from 0x69b4b45 to 0x69bc0b0, (0x756b bytes) 2024-09-13T09:28:01.348 INFO:teuthology.orchestra.run.smithi063.stdout:5848 writesame 0x7038ac5 thru 0x7039011 data_size 0x1(0x54d bytes) 2024-09-13T09:28:01.367 INFO:teuthology.orchestra.run.smithi063.stdout:5849 write 0xaa88bf0 thru 0xaa8e044 (0x5455 bytes) 2024-09-13T09:28:01.374 INFO:teuthology.orchestra.run.smithi063.stdout:5851 read 0x49cb760 thru 0x49d5383 (0x9c24 bytes) 2024-09-13T09:28:01.375 INFO:teuthology.orchestra.run.smithi063.stdout:5852 read 0xa462b0d thru 0xa4672fa (0x47ee bytes) 2024-09-13T09:28:01.375 INFO:teuthology.orchestra.run.smithi063.stdout:5853 read 0x852c78e thru 0x8535908 (0x917b bytes) 2024-09-13T09:28:01.393 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:01.393 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.652462640000003, 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.394 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:01.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:01.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:01.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141158 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 63488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.846060905, 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039207350000000002, 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141158 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544055+0000", 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544057+0000", 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8500000000000001e-06 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544062+0000", 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5940000000000004e-06 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544065+0000", 2024-09-13T09:28:01.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4570000000000001e-06 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583152+0000", 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039086837999999999 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583194+0000", 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2286999999999997e-05 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583263+0000", 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8569e-05 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141159 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 79872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.846047862000001, 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039321855000000003, 2024-09-13T09:28:01.405 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141159 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544068+0000", 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544070+0000", 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7719999999999999e-06 2024-09-13T09:28:01.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544074+0000", 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4209999999999997e-06 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544079+0000", 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3839999999999999e-06 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583292+0000", 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039213523 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583337+0000", 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5243e-05 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583390+0000", 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2769999999999998e-05 2024-09-13T09:28:01.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141160 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 120832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.846033851, 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039426997999999998, 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141160 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544082+0000", 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544085+0000", 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.605e-06 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544089+0000", 2024-09-13T09:28:01.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9609999999999996e-06 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544093+0000", 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9310000000000001e-06 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583414+0000", 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039320689999999998 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583455+0000", 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1267000000000003e-05 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583509+0000", 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4051999999999997e-05 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.414 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141161 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 137216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.846019123, 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039545311, 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141161 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544097+0000", 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544099+0000", 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.497e-06 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544105+0000", 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6080000000000003e-06 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544109+0000", 2024-09-13T09:28:01.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2440000000000004e-06 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583533+0000", 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039424167000000003 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583584+0000", 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0470000000000003e-05 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583642+0000", 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8644000000000003e-05 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141162 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 161792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.846002864999999, 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039683041000000002, 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141162 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544113+0000", 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544116+0000", 2024-09-13T09:28:01.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.464e-06 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544121+0000", 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.288e-06 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544125+0000", 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1389999999999997e-06 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583663+0000", 2024-09-13T09:28:01.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039537566000000003 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583747+0000", 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4695000000000001e-05 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583796+0000", 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9214e-05 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141163 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 178176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845987793000001, 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039773226000000002, 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141163 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544128+0000", 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544131+0000", 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.599e-06 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544152+0000", 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1195e-05 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544157+0000", 2024-09-13T09:28:01.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2229999999999998e-06 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583819+0000", 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039662206999999998 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583858+0000", 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8745999999999998e-05 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583902+0000", 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3553000000000001e-05 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141164 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845953958999999, 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039840280999999998, 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141164 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544162+0000", 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544165+0000", 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6599999999999999e-06 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544170+0000", 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6330000000000001e-06 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544174+0000", 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583924+0000", 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039749128000000002 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583962+0000", 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8461000000000003e-05 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584003+0000", 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0611000000000001e-05 2024-09-13T09:28:01.432 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141165 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 235520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845938393000001, 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039975881999999997, 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.433 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141165 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544178+0000", 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544179+0000", 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.891e-06 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544184+0000", 2024-09-13T09:28:01.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5510000000000001e-06 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544188+0000", 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8879999999999999e-06 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584060+0000", 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039871697999999997 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584106+0000", 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6199999999999998e-05 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584154+0000", 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7942000000000003e-05 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141166 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 260096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.437 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845924985, 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042489992999999997, 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141166 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544191+0000", 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544193+0000", 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9410000000000001e-06 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544198+0000", 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.04e-06 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544202+0000", 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0550000000000001e-06 2024-09-13T09:28:01.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584193+0000", 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.039990942000000002 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586600+0000", 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024071449999999999 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586681+0000", 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1153e-05 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141167 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 276480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845910774, 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042703159999999997, 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141167 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544205+0000", 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544207+0000", 2024-09-13T09:28:01.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.956e-06 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544212+0000", 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544216+0000", 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1819999999999999e-06 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586717+0000", 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042501039999999997 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586828+0000", 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000111097 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586909+0000", 2024-09-13T09:28:01.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0378999999999997e-05 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141168 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 317440~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544220+0000", 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845896584, 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042857105999999999, 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141168 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544220+0000", 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544220+0000", 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544219+0000", 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544221+0000", 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8190000000000001e-06 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544226+0000", 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544236+0000", 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4520000000000003e-06 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.586931+0000", 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042695205 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587002+0000", 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1062000000000005e-05 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587077+0000", 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4939999999999997e-05 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141169 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 333824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.451 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845876965, 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043015721999999999, 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141169 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544239+0000", 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544245+0000", 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544251+0000", 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4739999999999996e-06 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544255+0000", 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0829999999999997e-06 2024-09-13T09:28:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587125+0000", 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.042870220000000001 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587193+0000", 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7898999999999995e-05 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587255+0000", 2024-09-13T09:28:01.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1977999999999995e-05 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141170 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 358400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845857455999999, 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043172973000000003, 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.456 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141170 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544259+0000", 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544260+0000", 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8330000000000001e-06 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544265+0000", 2024-09-13T09:28:01.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0819999999999998e-06 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544270+0000", 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1439999999999996e-06 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587302+0000", 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043032778000000001 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587371+0000", 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8363999999999998e-05 2024-09-13T09:28:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587432+0000", 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1067000000000003e-05 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141171 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 374784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845843048999999, 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043357668000000002, 2024-09-13T09:28:01.460 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141171 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544273+0000", 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544275+0000", 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.962e-06 2024-09-13T09:28:01.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544280+0000", 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9180000000000002e-06 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544284+0000", 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1450000000000001e-06 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587466+0000", 2024-09-13T09:28:01.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043182470000000001 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587547+0000", 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0981000000000003e-05 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587631+0000", 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3473000000000005e-05 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141172 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 415744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845828937, 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043467455000000002, 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141172 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.465 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544287+0000", 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544289+0000", 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.88e-06 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544294+0000", 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544298+0000", 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1080000000000003e-06 2024-09-13T09:28:01.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587659+0000", 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043360797 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587711+0000", 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2472999999999997e-05 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587755+0000", 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3328000000000003e-05 2024-09-13T09:28:01.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141173 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 432128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544302+0000", 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845814517000001, 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043653346000000003, 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141173 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544302+0000", 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544302+0000", 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544301+0000", 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544303+0000", 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.978e-06 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544322+0000", 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8453000000000001e-05 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544327+0000", 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.707e-06 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587790+0000", 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043463357000000001 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587871+0000", 2024-09-13T09:28:01.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0987000000000002e-05 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.587955+0000", 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4158000000000003e-05 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141174 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 456704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845785269, 2024-09-13T09:28:01.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043818442999999999, 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141174 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544331+0000", 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544333+0000", 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.571e-06 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544339+0000", 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5049999999999998e-06 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544343+0000", 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8800000000000001e-06 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588002+0000", 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043659011999999997 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588091+0000", 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9777000000000005e-05 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588149+0000", 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8115999999999997e-05 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141175 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 473088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845770411, 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.044033668999999998, 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141175 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:28:01.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544346+0000", 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544348+0000", 2024-09-13T09:28:01.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6850000000000001e-06 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544353+0000", 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8330000000000001e-06 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544357+0000", 2024-09-13T09:28:01.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9040000000000002e-06 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588199+0000", 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043841822000000003 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588298+0000", 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8825000000000003e-05 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588380+0000", 2024-09-13T09:28:01.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1905999999999998e-05 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141176 3.7 3:e7cd5a29:::rbd_data.1118fafd09f3.000000000000010e:2a [read 514048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.845755833, 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.044145693999999999, 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141176 2024-09-13T09:28:01.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544360+0000", 2024-09-13T09:28:01.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544363+0000", 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4210000000000002e-06 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544367+0000", 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8409999999999999e-06 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.544372+0000", 2024-09-13T09:28:01.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.233e-06 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588414+0000", 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.044042247999999999 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588454+0000", 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9696000000000001e-05 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.588506+0000", 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2515000000000001e-05 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.486 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.487 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:01.488 INFO:teuthology.orchestra.run.smithi063.stdout:5854 write 0xe3fc462 thru 0xe401839 (0x53d8 bytes) 2024-09-13T09:28:01.489 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:01.489 INFO:teuthology.orchestra.run.smithi063.stdout:5855 trunc from 0xe40183a to 0x7b81144 2024-09-13T09:28:01.489 INFO:teuthology.orchestra.run.smithi063.stdout:5856 read 0x18d64ce thru 0x18dc7f7 (0x632a bytes) 2024-09-13T09:28:01.489 INFO:teuthology.orchestra.run.smithi063.stdout:5858 writesame 0x9faafcf thru 0x9fb8db8 data_size 0x41(0xddea bytes) 2024-09-13T09:28:01.489 INFO:teuthology.orchestra.run.smithi063.stdout:5859 read 0x7b390c8 thru 0x7b442e4 (0xb21d bytes) 2024-09-13T09:28:01.489 INFO:teuthology.orchestra.run.smithi063.stdout:5860 writesame 0x545f634 thru 0x54674a0 data_size 0x1(0x7e6d bytes) 2024-09-13T09:28:01.490 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:01.484+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:01.499 INFO:teuthology.orchestra.run.smithi063.stdout:5862 read 0x3035b4e thru 0x3036b6d (0x1020 bytes) 2024-09-13T09:28:01.500 INFO:teuthology.orchestra.run.smithi063.stdout:5863 read 0x6a1cb08 thru 0x6a23668 (0x6b61 bytes) 2024-09-13T09:28:01.501 INFO:teuthology.orchestra.run.smithi063.stdout:5866 write 0x581b0 thru 0x66ae8 (0xe939 bytes) 2024-09-13T09:28:01.517 INFO:teuthology.orchestra.run.smithi063.stdout:5869 trunc from 0x9fb8db9 to 0x4ba7173 2024-09-13T09:28:01.533 INFO:teuthology.orchestra.run.smithi063.stdout:5870 punch from 0x1a8a3c5 to 0x1a8fb57, (0x5792 bytes) 2024-09-13T09:28:01.543 INFO:teuthology.orchestra.run.smithi063.stdout:5871 trunc from 0x4ba7173 to 0x90f434d 2024-09-13T09:28:01.549 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:28:01.558 INFO:teuthology.orchestra.run.smithi063.stderr:set 3.18 pg_upmap mapping to [2,5] 2024-09-13T09:28:01.558 INFO:teuthology.orchestra.run.smithi063.stdout:5872 writesame 0x13865e6 thru 0x1393203 data_size 0x9(0xcc1e bytes) 2024-09-13T09:28:01.616 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:01.612+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:01.620 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:01.620 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:01.621 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:01.621 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:01.632 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:01.689 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:01.686+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:01.738 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:01.738 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:01.738 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:01.738 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:01.738 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:01.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:01.790 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:01.784+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:01.861 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 80.297050859999999, 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:01.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:01.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:95168 3.1 3:85864a9f:::rbd_data.1118298c62d6.0000000000000002:head [stat,write 2490368~4977] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 71.589481578000004, 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029798249999999998, 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.867 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 95168 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269662+0000", 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269673+0000", 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1345999999999999e-05 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269682+0000", 2024-09-13T09:28:01.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8829999999999995e-06 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269689+0000", 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2180000000000002e-06 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269756+0000", 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6506000000000003e-05 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.269934+0000", 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000178159 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.270171+0000", 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023732199999999999 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.271368+0000", 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001197128 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299354+0000", 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.027986021999999999 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299402+0000", 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8254999999999998e-05 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:50.299460+0000", 2024-09-13T09:28:01.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7988000000000001e-05 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.709392481000002, 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.873 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:01.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130769 3.1 3:84b2050c:::rbd_data.1118de6ae5af.0000000000000120:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 32.824038733000002, 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024401849, 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130769 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035105+0000", 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035103+0000", 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035114+0000", 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1613e-05 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035136+0000", 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1472e-05 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035145+0000", 2024-09-13T09:28:01.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4860000000000003e-06 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035204+0000", 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9027000000000003e-05 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.035363+0000", 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015868600000000001 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 3", 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.040402+0000", 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050390000000000001 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059321+0000", 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018919062 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059340+0000", 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9001999999999999e-05 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059432+0000", 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1535999999999999e-05 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:29.059507+0000", 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5220000000000001e-05 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139806 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 309248~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473237246, 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022069070999999999, 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139806 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.884 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385906+0000", 2024-09-13T09:28:01.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385909+0000", 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.543e-06 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385914+0000", 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.789e-06 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385919+0000", 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.899e-06 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.407837+0000", 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02191802 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.407916+0000", 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8829e-05 2024-09-13T09:28:01.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.407976+0000", 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9453999999999997e-05 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139807 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473220236, 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022177722, 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139807 2024-09-13T09:28:01.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385923+0000", 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385926+0000", 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.424e-06 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385933+0000", 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7340000000000008e-06 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385938+0000", 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8199999999999996e-06 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408007+0000", 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022068910000000001 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408062+0000", 2024-09-13T09:28:01.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5324000000000001e-05 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408101+0000", 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8840000000000001e-05 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139808 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385943+0000", 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473201096, 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022281553999999999, 2024-09-13T09:28:01.893 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139808 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385943+0000", 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385943+0000", 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385942+0000", 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385945+0000", 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8150000000000002e-06 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385951+0000", 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0170000000000002e-06 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385956+0000", 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7210000000000002e-06 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408116+0000", 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022160119999999998 2024-09-13T09:28:01.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408177+0000", 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0878000000000002e-05 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408224+0000", 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7355999999999999e-05 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139809 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 391168~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473183634, 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022375121000000001, 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139809 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385960+0000", 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385962+0000", 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.272e-06 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385975+0000", 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2604e-05 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.385986+0000", 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1049e-05 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408258+0000", 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022272734999999998 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408299+0000", 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0599999999999998e-05 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408335+0000", 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6186000000000002e-05 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139810 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 407552~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386011+0000", 2024-09-13T09:28:01.902 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473132786000001, 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022448594999999998, 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139810 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386011+0000", 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386011+0000", 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386010+0000", 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386014+0000", 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1769999999999998e-06 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386026+0000", 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2289999999999999e-05 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386036+0000", 2024-09-13T09:28:01.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0244000000000001e-05 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408349+0000", 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022313146999999998 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408418+0000", 2024-09-13T09:28:01.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9099999999999999e-05 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408460+0000", 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1156999999999999e-05 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139811 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 448512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.47310154, 2024-09-13T09:28:01.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022522297, 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139811 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386042+0000", 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386045+0000", 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8420000000000001e-06 2024-09-13T09:28:01.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386051+0000", 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386056+0000", 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8100000000000003e-06 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408474+0000", 2024-09-13T09:28:01.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022417245999999998 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408528+0000", 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4861999999999998e-05 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408564+0000", 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6022999999999997e-05 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139812 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 464896~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473082471, 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022630847999999999, 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139812 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386061+0000", 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386064+0000", 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6919999999999998e-06 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386070+0000", 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2040000000000002e-06 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386075+0000", 2024-09-13T09:28:01.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9640000000000002e-06 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408581+0000", 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02250628 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408653+0000", 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2151999999999996e-05 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408692+0000", 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8923e-05 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139813 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 489472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473064527, 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022729668000000001, 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.916 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139813 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386079+0000", 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386081+0000", 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:01.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386087+0000", 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7200000000000003e-06 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386092+0000", 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0540000000000002e-06 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408710+0000", 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022617624999999999 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408762+0000", 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2592000000000001e-05 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408809+0000", 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6519e-05 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139814 3.6 3:67f4f206:::rbd_data.1118fafd09f3.0000000000000004:2a [read 505856~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473047287, 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022830249, 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139814 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386096+0000", 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386105+0000", 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7819999999999999e-06 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386112+0000", 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6880000000000002e-06 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386117+0000", 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:28:01.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408823+0000", 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022706271 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408886+0000", 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2645999999999996e-05 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408927+0000", 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0911999999999997e-05 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139815 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386122+0000", 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473021648, 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029220356999999999, 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.925 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139815 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386122+0000", 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386122+0000", 2024-09-13T09:28:01.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386121+0000", 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386125+0000", 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7519999999999998e-06 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386131+0000", 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386136+0000", 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0379999999999999e-06 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.408941+0000", 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.022804334999999998 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409017+0000", 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6668999999999994e-05 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409287+0000", 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00026946099999999997 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413128+0000", 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003840979 2024-09-13T09:28:01.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415236+0000", 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021087279999999998 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415293+0000", 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6660999999999998e-05 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415342+0000", 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9279e-05 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139816 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.473002562, 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.02928824, 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139816 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386141+0000", 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386144+0000", 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6119999999999999e-06 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386150+0000", 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0719999999999996e-06 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386155+0000", 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9779999999999996e-06 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409919+0000", 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.023763375999999999 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.409981+0000", 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2708999999999999e-05 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.410214+0000", 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023281800000000001 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413143+0000", 2024-09-13T09:28:01.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0029293129999999998 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415351+0000", 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022077099999999999 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415391+0000", 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9702e-05 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415429+0000", 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8494000000000001e-05 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139817 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386160+0000", 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.472984153000001, 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.029572294999999998, 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139817 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386160+0000", 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386160+0000", 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386159+0000", 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386162+0000", 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9869999999999999e-06 2024-09-13T09:28:01.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386168+0000", 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4110000000000004e-06 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386172+0000", 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.809e-06 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.410784+0000", 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.024611091000000002 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.410848+0000", 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4542999999999995e-05 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.411102+0000", 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025437799999999998 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413162+0000", 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020593289999999999 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415668+0000", 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0025066630000000001 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415698+0000", 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9192000000000001e-05 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415732+0000", 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4239999999999997e-05 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139818 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.472966812999999, 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030114272000000001, 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.943 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139818 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:28:01.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386177+0000", 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386180+0000", 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1259999999999998e-06 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386185+0000", 2024-09-13T09:28:01.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5790000000000003e-06 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386190+0000", 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1499999999999998e-06 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.411771+0000", 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025581115000000001 2024-09-13T09:28:01.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.411843+0000", 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1823999999999995e-05 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412077+0000", 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023346400000000001 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415093+0000", 2024-09-13T09:28:01.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003016482 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416209+0000", 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011161369999999999 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416256+0000", 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6456999999999999e-05 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416291+0000", 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5330999999999998e-05 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139819 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.472948504, 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.030190423000000001, 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.949 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139819 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386195+0000", 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386205+0000", 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0128e-05 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386211+0000", 2024-09-13T09:28:01.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5799999999999997e-06 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.386216+0000", 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.899e-06 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412527+0000", 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.026310579000000001 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412573+0000", 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6637000000000001e-05 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.412749+0000", 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017502199999999999 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415137+0000", 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0023887359999999998 2024-09-13T09:28:01.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416302+0000", 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001165234 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416347+0000", 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4428000000000002e-05 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416386+0000", 2024-09-13T09:28:01.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8733000000000003e-05 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139820 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.393356+0000", 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.465787487, 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.023103754000000001, 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.955 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139820 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393356+0000", 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393356+0000", 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393354+0000", 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393365+0000", 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0968000000000001e-05 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393379+0000", 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3825000000000001e-05 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.393393+0000", 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3900000000000001e-05 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413174+0000", 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019781499000000001 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413218+0000", 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3829000000000003e-05 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.413500+0000", 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028210300000000002 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.415552+0000", 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020520780000000001 2024-09-13T09:28:01.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416396+0000", 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00084346599999999996 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416430+0000", 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4542999999999998e-05 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416460+0000", 2024-09-13T09:28:01.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9802e-05 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:139821 3.6 3:6175e419:::rbd_object_map.11181cfec05f:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.394603+0000", 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.464540362999999, 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.021986615000000001, 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:01.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 139821 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394603+0000", 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394603+0000", 2024-09-13T09:28:01.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394602+0000", 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394608+0000", 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6230000000000002e-06 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394622+0000", 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3963e-05 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.394630+0000", 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2779999999999993e-06 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.414023+0000", 2024-09-13T09:28:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.019392857999999999 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.414111+0000", 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7719000000000005e-05 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 1", 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.414481+0000", 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00036972300000000002 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416165+0000", 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016848169999999999 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416473+0000", 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00030729999999999999 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416546+0000", 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3529999999999996e-05 2024-09-13T09:28:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.416590+0000", 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3683000000000002e-05 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:01.967 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:01.969 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:01.969 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:01.962+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:02.070 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:02.064+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:02.125 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:02.126 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:02.126 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:02.126 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:02.136 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:02.184 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:02.180+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:02.246 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:02.247 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:02.247 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:02.247 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:02.247 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:02.256 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:02.272 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:02.270+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:02.365 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784802693, 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784790456, 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784775058999999, 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784760184, 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784742975, 2024-09-13T09:28:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784722520000001, 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784704440000001, 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784688064000001, 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784672712999999, 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784656009000001, 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784637927, 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784621182, 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.784576523, 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.781627083, 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.781606545000001, 2024-09-13T09:28:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.78158393, 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.781565062, 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.781530468, 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.777251563, 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.777187591000001, 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.545 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:02.547 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:02.547 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:02.542+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:02.564 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:28:02.647 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:02.642+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:02.663 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:02.663 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:02.663 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:02.663 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:02.673 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:02.748 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:02.742+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:02.782 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:02.782 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:02.783 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:02.783 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:02.783 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:02.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:02.850 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:02.844+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:02.898 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:02.898 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:02.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:4603 3.4 e35/21 3:24d7ded9:::rbd_data.11189fb5ddd6.0000000000000005:head v 35'104, mlcod=35'102)", 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.929603196, 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018576262999999999, 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966133+0000", 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.966131+0000", 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976375+0000", 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.010243188 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976401+0000", 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6792999999999999e-05 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976410+0000", 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9889999999999998e-06 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976463+0000", 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2471e-05 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.976488+0000", 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5062000000000002e-05 2024-09-13T09:28:02.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984558+0000", 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0080697229999999991 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:09.984709+0000", 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015165 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.903 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:42978 3.0 e56/21 3:07f92e03:::rbd_data.111838d2a860.0000000000000001:head v 56'148, mlcod=55'146)", 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.263559310000005, 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.014933016, 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632177+0000", 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.632175+0000", 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637767+0000", 2024-09-13T09:28:02.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055920939999999997 2024-09-13T09:28:02.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637797+0000", 2024-09-13T09:28:02.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9380999999999999e-05 2024-09-13T09:28:02.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637810+0000", 2024-09-13T09:28:02.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2867000000000001e-05 2024-09-13T09:28:02.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637870+0000", 2024-09-13T09:28:02.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.016e-05 2024-09-13T09:28:02.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.637905+0000", 2024-09-13T09:28:02.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4941000000000003e-05 2024-09-13T09:28:02.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647043+0000", 2024-09-13T09:28:02.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0091382979999999996 2024-09-13T09:28:02.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.647110+0000", 2024-09-13T09:28:02.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6995000000000005e-05 2024-09-13T09:28:02.912 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.912 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.912 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:44939 3.9 e56/21 3:933c56d6:::rbd_data.111838d2a860.0000000000000000:head v 56'109, mlcod=55'107)", 2024-09-13T09:28:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:28:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.058533261999997, 2024-09-13T09:28:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.025371786, 2024-09-13T09:28:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:28:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.837203+0000", 2024-09-13T09:28:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855615+0000", 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.018412867999999999 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855647+0000", 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1347000000000003e-05 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855660+0000", 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2931e-05 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855732+0000", 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2589999999999994e-05 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.855763+0000", 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0630999999999999e-05 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862508+0000", 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0067450640000000003 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:37.862575+0000", 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6727e-05 2024-09-13T09:28:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:75345 3.0 e62/21 3:02e17839:::rbd_data.11181839c212.0000000000000076:head v 62'158, mlcod=62'156)", 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.884794038999999, 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047207880000000001, 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010942+0000", 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010940+0000", 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010954+0000", 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.396e-05 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.010984+0000", 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9711000000000001e-05 2024-09-13T09:28:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011001+0000", 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6515e-05 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011163+0000", 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016227 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.011253+0000", 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0321000000000005e-05 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015592+0000", 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0043389919999999998 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.015663+0000", 2024-09-13T09:28:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0743999999999999e-05 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:77475 3.0 e62/21 3:05ac9e4e:::rbd_data.11181839c212.0000000000000178:head v 62'161, mlcod=62'159)", 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 76.690711596, 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050650920000000002, 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205025+0000", 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205023+0000", 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205066+0000", 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2531999999999999e-05 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205095+0000", 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9428999999999999e-05 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205107+0000", 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2092e-05 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205253+0000", 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014548799999999999 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.205283+0000", 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0077e-05 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210026+0000", 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047436680000000004 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:46.210090+0000", 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3281999999999995e-05 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:88022 3.0 e63/21 3:0521cabd:::rbd_data.11181839c212.00000000000002dd:head v 63'172, mlcod=63'170)", 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 75.306348755000002, 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045245789999999999, 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589387+0000", 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589385+0000", 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589404+0000", 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9077e-05 2024-09-13T09:28:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589442+0000", 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7610000000000001e-05 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589460+0000", 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8216999999999999e-05 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589579+0000", 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118603 2024-09-13T09:28:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.589637+0000", 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8780999999999998e-05 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593854+0000", 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0042163610000000001 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:47.593912+0000", 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8236999999999998e-05 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:128966 3.9 3:9420bb05:::rbd_data.11181500dc3e.000000000000008f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.006985426, 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053551420000000002, 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 128966 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888751+0000", 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888748+0000", 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888764+0000", 2024-09-13T09:28:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6772000000000001e-05 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888807+0000", 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2827000000000002e-05 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888829+0000", 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1426000000000001e-05 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.888965+0000", 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013610599999999999 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889174+0000", 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00020964700000000001 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.889456+0000", 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000281403 2024-09-13T09:28:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.890649+0000", 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011928819999999999 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.893917+0000", 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003268328 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894020+0000", 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103003 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:24.894106+0000", 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5986e-05 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:130683 3.9 3:9160b546:::rbd_data.1118de6ae5af.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-13T09:28:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.085015232000003, 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005777351, 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 130683 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810721+0000", 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.810718+0000", 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999967 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811313+0000", 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00059520999999999999 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811352+0000", 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9453999999999999e-05 2024-09-13T09:28:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811369+0000", 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6886999999999999e-05 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811475+0000", 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105845 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.811814+0000", 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00033890599999999998 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.812483+0000", 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00066937099999999999 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816386+0000", 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0039026949999999999 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816411+0000", 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4916e-05 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816465+0000", 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3835000000000002e-05 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:28.816498+0000", 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3555e-05 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136071 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'381, mlcod=111'376)", 2024-09-13T09:28:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:28:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.085437824, 2024-09-13T09:28:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048782419999999996, 2024-09-13T09:28:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:28:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:28:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810298+0000", 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810306+0000", 2024-09-13T09:28:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3920000000000003e-06 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810322+0000", 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6520000000000001e-05 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810331+0000", 2024-09-13T09:28:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4160000000000004e-06 2024-09-13T09:28:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810388+0000", 2024-09-13T09:28:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7163999999999997e-05 2024-09-13T09:28:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.810414+0000", 2024-09-13T09:28:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6163999999999999e-05 2024-09-13T09:28:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815117+0000", 2024-09-13T09:28:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047025620000000004 2024-09-13T09:28:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.815177+0000", 2024-09-13T09:28:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9892999999999997e-05 2024-09-13T09:28:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136080 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'385, mlcod=111'382)", 2024-09-13T09:28:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:28:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.033203509, 2024-09-13T09:28:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0057524539999999997, 2024-09-13T09:28:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:28:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862533+0000", 2024-09-13T09:28:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862531+0000", 2024-09-13T09:28:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862543+0000", 2024-09-13T09:28:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1538e-05 2024-09-13T09:28:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862560+0000", 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7156e-05 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862572+0000", 2024-09-13T09:28:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1606e-05 2024-09-13T09:28:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862640+0000", 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8695999999999994e-05 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.862669+0000", 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8354999999999999e-05 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868211+0000", 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055418450000000001 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.868285+0000", 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4486000000000005e-05 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136101 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'391, mlcod=111'388)", 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.989963093, 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050269909999999998, 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905773+0000", 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905772+0000", 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905783+0000", 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0822000000000001e-05 2024-09-13T09:28:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905796+0000", 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3328e-05 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905805+0000", 2024-09-13T09:28:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6079999999999996e-06 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905898+0000", 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.2958e-05 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.905922+0000", 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4572000000000001e-05 2024-09-13T09:28:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910718+0000", 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047962489999999998 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.910800+0000", 2024-09-13T09:28:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1749999999999995e-05 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:136103 3.4 e111/21 3:2346cbdd:::rbd_object_map.1118d1656b7c:head v 111'393, mlcod=111'388)", 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.987594282, 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0052766970000000003, 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908142+0000", 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908141+0000", 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908153+0000", 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2686e-05 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908170+0000", 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6651999999999999e-05 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908187+0000", 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908236+0000", 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8959000000000003e-05 2024-09-13T09:28:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.908275+0000", 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8983999999999998e-05 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913370+0000", 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050951190000000004 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:43.913419+0000", 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8293e-05 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:138187 3.0 e113/21 3:05ffe32b:::rbd_data.11181cfec05f.000000000000001c:head v 113'771, mlcod=113'769)", 2024-09-13T09:28:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.602862368, 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051940129999999999, 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292874+0000", 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292873+0000", 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292894+0000", 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1759999999999998e-05 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292910+0000", 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5707000000000001e-05 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292917+0000", 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9909999999999996e-06 2024-09-13T09:28:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292970+0000", 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2494999999999998e-05 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.292996+0000", 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6027e-05 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.298020+0000", 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0050239480000000003 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.298068+0000", 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8189999999999998e-05 2024-09-13T09:28:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:141269 3.0 e114/21 3:03e78e0b:::rbd_data.11181cfec05f.00000000000000ed:head v 114'774, mlcod=114'772)", 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.283220519, 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053503429999999996, 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612516+0000", 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612515+0000", 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612522+0000", 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4320000000000004e-06 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612530+0000", 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7579999999999998e-06 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612536+0000", 2024-09-13T09:28:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9560000000000002e-06 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612626+0000", 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9307000000000004e-05 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.612646+0000", 2024-09-13T09:28:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0349000000000001e-05 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.617820+0000", 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.005173731 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.617866+0000", 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6325000000000001e-05 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:143378 3.4 e115/21 3:232b5a0d:::rbd_data.11181cfec05f.00000000000000f2:head v 115'577, mlcod=115'574)", 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:48.736295+0000", 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 14.159441170999999, 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0062177999999999999, 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736295+0000", 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736295+0000", 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736294+0000", 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736361+0000", 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6675999999999997e-05 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736381+0000", 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0720000000000002e-05 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736389+0000", 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.089e-06 2024-09-13T09:28:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736450+0000", 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0544000000000002e-05 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.736477+0000", 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7608000000000001e-05 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.742468+0000", 2024-09-13T09:28:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0059905610000000001 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:48.742513+0000", 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4790999999999999e-05 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:157693 3.0 e118/21 3:0048f95a:::rbd_data.11183cfd6e74.000000000000029a:head v 118'821, mlcod=118'819)", 2024-09-13T09:28:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:52.605390+0000", 2024-09-13T09:28:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 10.290346014000001, 2024-09-13T09:28:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0053676189999999997, 2024-09-13T09:28:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605390+0000", 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605390+0000", 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605389+0000", 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605399+0000", 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8740000000000007e-06 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605417+0000", 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7968000000000001e-05 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605425+0000", 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9200000000000004e-06 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605495+0000", 2024-09-13T09:28:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0158000000000002e-05 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.605514+0000", 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.874e-05 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.610704+0000", 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0051900399999999999 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:52.610758+0000", 2024-09-13T09:28:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4166999999999998e-05 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:158259 3.4 e118/21 3:25a994f0:::rbd_data.11183cfd6e74.0000000000000158:head v 118'641, mlcod=118'638)", 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.200972+0000", 2024-09-13T09:28:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6947642569999992, 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0048042680000000004, 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.200972+0000", 2024-09-13T09:28:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.200972+0000", 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.200971+0000", 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.200981+0000", 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3780000000000007e-06 2024-09-13T09:28:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.200989+0000", 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7169999999999997e-06 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.200995+0000", 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5389999999999998e-06 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.201031+0000", 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6529999999999998e-05 2024-09-13T09:28:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.201055+0000", 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3351999999999999e-05 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.205733+0000", 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0046778660000000001 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.205776+0000", 2024-09-13T09:28:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3574e-05 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:158711 3.9 e119/107 3:902a30d7:::rbd_data.11183cfd6e74.00000000000001a7:head v 119'349, mlcod=119'346)", 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.612908+0000", 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.2828284990000007, 2024-09-13T09:28:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0056662429999999996, 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.612908+0000", 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.612908+0000", 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.612907+0000", 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.612912+0000", 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:28:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.612920+0000", 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6499999999999996e-06 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.612926+0000", 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9359999999999996e-06 2024-09-13T09:28:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.613575+0000", 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00064922999999999999 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.613599+0000", 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3390000000000001e-05 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.618537+0000", 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0049387509999999999 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.618574+0000", 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6482000000000001e-05 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:158931 3.0 e119/21 3:02493eda:::rbd_data.11183cfd6e74.00000000000000ed:head v 119'849, mlcod=119'847)", 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:53.890154+0000", 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0055819509999999, 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0045950840000000001, 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890154+0000", 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890154+0000", 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890154+0000", 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890161+0000", 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.126e-06 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890170+0000", 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6069999999999997e-06 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890177+0000", 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6059999999999999e-06 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890240+0000", 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3244999999999995e-05 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.890259+0000", 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8777e-05 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.894705+0000", 2024-09-13T09:28:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0044460769999999997 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:53.894749+0000", 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4363999999999997e-05 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.3372766730000001, 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:03.022 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:03.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:03.025 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:03.022+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:03.125 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:03.120+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:03.139 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:03.139 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:03.140 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:03.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:03.226 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:03.222+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:03.256 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:03.256 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:03.256 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:03.256 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:03.256 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:03.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:03.327 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:03.320+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:03.351 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:03.372 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:03.372 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:03.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:03.372 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675636424000004, 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:28:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675613909999996, 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:28:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675598073000003, 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:28:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675401128999994, 2024-09-13T09:28:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:28:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675376804999999, 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675360345000001, 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:28:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675344323999994, 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:28:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:28:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675293991000004, 2024-09-13T09:28:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:28:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.675190416000007, 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:28:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:28:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665812619999997, 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:28:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665793956000002, 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:28:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:28:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665777465999994, 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:28:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665761681000006, 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:28:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665745560000005, 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:28:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:28:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665729655000007, 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:28:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:28:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665713186000005, 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:28:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:28:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665691334000002, 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:28:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:28:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665674519000007, 2024-09-13T09:28:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:28:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:28:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665658222000005, 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:28:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.665642621000003, 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:03.462 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:03.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:03.463 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:03.456+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:03.507 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:03.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:03.554 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:03.563 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:03.558+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:03.575 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:03.575 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:03.575 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:03.581 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:03.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:03.602 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:03.605 INFO:teuthology.orchestra.run.smithi063.stdout:5873 write 0x20744ae thru 0x207e700 (0xa253 bytes) 2024-09-13T09:28:03.613 INFO:teuthology.orchestra.run.smithi063.stdout:5878 trunc from 0x90f434d to 0x6f9939d 2024-09-13T09:28:03.626 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:03.629 INFO:teuthology.orchestra.run.smithi063.stdout:5880 trunc from 0x6f9939d to 0x7ecc81f 2024-09-13T09:28:03.650 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:03.657 INFO:teuthology.orchestra.run.smithi063.stdout:5881 write 0xa8439cf thru 0xa853190 (0xf7c2 bytes) 2024-09-13T09:28:03.664 INFO:teuthology.orchestra.run.smithi063.stdout:5882 punch from 0x97b1b18 to 0x97bfc93, (0xe17b bytes) 2024-09-13T09:28:03.664 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:03.660+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:03.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:03.676 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:03.677 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:03.677 INFO:teuthology.orchestra.run.smithi063.stdout:5883 write 0xc9de0e1 thru 0xc9e04cb (0x23eb bytes) 2024-09-13T09:28:03.692 INFO:teuthology.orchestra.run.smithi063.stdout:5884 trunc from 0xc9e04cc to 0x76b2db9 2024-09-13T09:28:03.705 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:03.705 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:03.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:03.735 INFO:teuthology.orchestra.run.smithi063.stdout:5886 writesame 0xdf6f94b thru 0xdf6fd4b data_size 0x1(0x401 bytes) 2024-09-13T09:28:03.735 INFO:teuthology.orchestra.run.smithi063.stdout:5888 writesame 0x40c0d42 thru 0x40cdc6b data_size 0x3(0xcf2a bytes) 2024-09-13T09:28:03.748 INFO:teuthology.orchestra.run.smithi063.stdout:5891 write 0xbd1d69a thru 0xbd2aac7 (0xd42e bytes) 2024-09-13T09:28:03.754 INFO:teuthology.orchestra.run.smithi063.stdout:5892 writesame 0x962317b thru 0x9626e9e data_size 0x56(0x3d24 bytes) 2024-09-13T09:28:03.758 INFO:teuthology.orchestra.run.smithi063.stdout:5893 read 0x2d68c76 thru 0x2d6a184 (0x150f bytes) 2024-09-13T09:28:03.759 INFO:teuthology.orchestra.run.smithi063.stdout:5894 read 0x1be0347 thru 0x1be1c78 (0x1932 bytes) 2024-09-13T09:28:03.759 INFO:teuthology.orchestra.run.smithi063.stdout:5895 read 0x7f3bf2c thru 0x7f4b0bd (0xf192 bytes) 2024-09-13T09:28:03.760 INFO:teuthology.orchestra.run.smithi063.stdout:5897 read 0xa690e29 thru 0xa6965fc (0x57d4 bytes) 2024-09-13T09:28:03.765 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:03.762+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:03.768 INFO:teuthology.orchestra.run.smithi063.stdout:5899 trunc from 0xdf6fd4c to 0xab38812 2024-09-13T09:28:03.785 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:03.785 INFO:teuthology.orchestra.run.smithi063.stdout:5901 read 0x46d95cd thru 0x46dfb09 (0x653d bytes) 2024-09-13T09:28:03.786 INFO:teuthology.orchestra.run.smithi063.stdout:5902 punch from 0x6453e97 to 0x645e010, (0xa179 bytes) 2024-09-13T09:28:03.786 INFO:teuthology.orchestra.run.smithi063.stdout:5903 writesame 0x753e349 thru 0x754114a data_size 0x27(0x2e02 bytes) 2024-09-13T09:28:03.789 INFO:teuthology.orchestra.run.smithi063.stdout:5904 read 0x9389b53 thru 0x938dc1c (0x40ca bytes) 2024-09-13T09:28:03.791 INFO:teuthology.orchestra.run.smithi063.stdout:5905 trunc from 0xab38812 to 0xb72c1af 2024-09-13T09:28:03.802 INFO:teuthology.orchestra.run.smithi063.stdout:5906 read 0x788cae1 thru 0x7891862 (0x4d82 bytes) 2024-09-13T09:28:03.803 INFO:teuthology.orchestra.run.smithi063.stdout:5907 writesame 0xa6b380c thru 0xa6bf66b data_size 0x10(0xbe60 bytes) 2024-09-13T09:28:03.812 INFO:teuthology.orchestra.run.smithi063.stdout:5909 read 0x36e7a0d thru 0x36e9d4c (0x2340 bytes) 2024-09-13T09:28:03.814 INFO:teuthology.orchestra.run.smithi063.stdout:5910 read 0x7e4b2e3 thru 0x7e4df0d (0x2c2b bytes) 2024-09-13T09:28:03.823 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:03.824 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:03.825 INFO:teuthology.orchestra.run.smithi063.stdout:5911 trunc from 0xb72c1af to 0x703a2ef 2024-09-13T09:28:03.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:03.854 INFO:teuthology.orchestra.run.smithi063.stdout:5912 write 0x385f890 thru 0x385f9a1 (0x112 bytes) 2024-09-13T09:28:03.878 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:03.874+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:03.884 INFO:teuthology.orchestra.run.smithi063.stdout:5914 write 0xce5f4e1 thru 0xce61e53 (0x2973 bytes) 2024-09-13T09:28:03.890 INFO:teuthology.orchestra.run.smithi063.stdout:5915 write 0xa2ec820 thru 0xa2ecb26 (0x307 bytes) 2024-09-13T09:28:03.895 INFO:teuthology.orchestra.run.smithi063.stdout:5916 write 0x376315b thru 0x3764c63 (0x1b09 bytes) 2024-09-13T09:28:03.901 INFO:teuthology.orchestra.run.smithi063.stdout:5917 trunc from 0xce61e54 to 0xe54ca82 2024-09-13T09:28:03.911 INFO:teuthology.orchestra.run.smithi063.stdout:5918 clone 45 order 19 su 2048 sc 10 2024-09-13T09:28:03.967 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:03.962+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:03.988 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:03.988 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:03.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:04.068 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:04.062+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:04.111 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.112 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.122 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:04.169 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:04.164+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:04.270 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:04.264+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:04.279 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.279 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.289 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:04.371 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:04.364+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:04.393 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.393 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:04.473 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:04.466+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:04.509 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.510 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:04.573 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:04.570+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:04.580 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:28:04.668 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.669 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.674 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:04.670+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:04.678 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:04.775 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:04.768+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:04.790 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.790 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:04.876 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:04.872+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:04.916 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:04.916 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:04.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:04.977 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:04.972+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:05.075 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:05.075 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:05.078 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:05.072+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:05.085 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:05.178 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:05.172+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:05.242 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:05.242 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:05.253 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:05.279 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:05.274+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:05.357 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:05.357 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:05.367 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:05.379 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:05.372+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:05.476 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:05.477 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:05.480 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:05.476+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:05.486 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:05.582 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:05.574+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:05.585 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:28:05.649 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:05.649 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:05.659 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:05.670 INFO:teuthology.orchestra.run.smithi063.stdout:truncating image image_client.0-clone44 from 0xe54ca82 (overlap 0x1fe703c) to 0xff381e 2024-09-13T09:28:05.681 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:05.676+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:05.691 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:05.770 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:05.770 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:05.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:05.782 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:05.776+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:05.833 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:05.854 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:05.877 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:05.883 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:05.878+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.886 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.307045887000001, 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:05.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:05.890 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.307033650000001, 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:05.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:05.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.307018252999999, 2024-09-13T09:28:05.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.899 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.307003378000001, 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.900 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:05.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306986168999998, 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:05.905 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306965714, 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:05.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:05.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.914 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306947634, 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:05.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:05.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306931257999999, 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.919 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:05.923 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306915907, 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:05.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306899203, 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:05.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306881121, 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.932 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:05.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:05.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306864376, 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:05.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:05.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:05.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.942 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.306819717, 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:05.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:05.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.303870277000001, 2024-09-13T09:28:05.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:05.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:05.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.303849739, 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.952 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:05.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:05.956 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.303827124000001, 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:05.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:05.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:05.961 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.303808256, 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:05.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:05.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.303773662000001, 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.966 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:05.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.299494757000001, 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.974 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.299430784999998, 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:05.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:05.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:05.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:05.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:05.979 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:05.981 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:05.981 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:05.981 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-13T09:28:05.982 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:05.982 INFO:teuthology.orchestra.run.smithi063.stdout:5919 write 0xb1108b8 thru 0xb114fa2 (0x46eb bytes) 2024-09-13T09:28:05.982 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:05.982 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:05.984 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:05.980+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:06.000 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:06.002 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:06.002 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:06.002 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:06.002 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:06.003 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:06.003 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:06.003 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:06.003 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:06.074 INFO:teuthology.orchestra.run.smithi063.stdout:5920 write 0xd30ee51 thru 0xd314038 (0x51e8 bytes) 2024-09-13T09:28:06.084 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:06.078+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:06.115 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:06.115 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:06.115 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:06.116 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:06.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:06.185 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:06.180+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:06.208 INFO:teuthology.orchestra.run.smithi063.stdout:5921 trunc from 0xe54ca82 to 0x93e2060 2024-09-13T09:28:06.262 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:06.263 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:06.263 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:06.263 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:06.263 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:06.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:06.286 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:06.282+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:06.365 INFO:teuthology.orchestra.run.smithi063.stdout:5922 trunc from 0x93e2060 to 0xa13883d 2024-09-13T09:28:06.375 INFO:teuthology.orchestra.run.smithi063.stdout:5927 read 0xa269c9 thru 0xa2d5bd (0x6bf5 bytes) 2024-09-13T09:28:06.387 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:06.382+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.8641757739999996, 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:06.426 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:06.430 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171288 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30572544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287079+0000", 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13555668600000001, 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046167272000000002, 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.431 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171288 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287079+0000", 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287079+0000", 2024-09-13T09:28:06.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287078+0000", 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287081+0000", 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.225e-06 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287102+0000", 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.054e-05 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287108+0000", 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1229999999999997e-06 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333099+0000", 2024-09-13T09:28:06.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.045990757 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333184+0000", 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5372999999999997e-05 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333246+0000", 2024-09-13T09:28:06.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1766999999999993e-05 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171292 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30654464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287680+0000", 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13495549600000001, 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046246486000000003, 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.436 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171292 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287680+0000", 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287680+0000", 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287680+0000", 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287681+0000", 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4810000000000001e-06 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287684+0000", 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8540000000000001e-06 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287686+0000", 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333766+0000", 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046080059999999999 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333854+0000", 2024-09-13T09:28:06.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7255000000000004e-05 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333926+0000", 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2770999999999998e-05 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171293 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30674944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287689+0000", 2024-09-13T09:28:06.441 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13494611200000001, 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046401347000000003, 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171293 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287689+0000", 2024-09-13T09:28:06.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287689+0000", 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287689+0000", 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287690+0000", 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3599999999999999e-06 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287693+0000", 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7089999999999998e-06 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287709+0000", 2024-09-13T09:28:06.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6308e-05 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.333954+0000", 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046245043999999999 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334048+0000", 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3665999999999994e-05 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334091+0000", 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2472000000000003e-05 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171294 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30695424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287715+0000", 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.134920706, 2024-09-13T09:28:06.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046545494999999999, 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171294 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287715+0000", 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287715+0000", 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287714+0000", 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287723+0000", 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5210000000000002e-06 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287730+0000", 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2919999999999998e-06 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287735+0000", 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6149999999999996e-06 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334123+0000", 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046387181999999999 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334210+0000", 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7251999999999997e-05 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334260+0000", 2024-09-13T09:28:06.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0238999999999998e-05 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171295 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30715904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287740+0000", 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13489506100000001, 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046690015000000001, 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171295 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287740+0000", 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287740+0000", 2024-09-13T09:28:06.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287740+0000", 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287742+0000", 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6340000000000001e-06 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287748+0000", 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287753+0000", 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4259999999999996e-06 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334288+0000", 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046535452999999997 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334374+0000", 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.5552999999999999e-05 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334430+0000", 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6654999999999999e-05 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171296 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30736384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287757+0000", 2024-09-13T09:28:06.456 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.134877989, 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046831794000000003, 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171296 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287757+0000", 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287757+0000", 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287757+0000", 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287760+0000", 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287766+0000", 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1240000000000002e-06 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287770+0000", 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4769999999999997e-06 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334455+0000", 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046684730000000001 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334530+0000", 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5307000000000003e-05 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334589+0000", 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8885999999999998e-05 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171297 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30756864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287774+0000", 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13486088299999999, 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046986285000000003, 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.461 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171297 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287774+0000", 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287774+0000", 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287774+0000", 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287783+0000", 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4829999999999999e-06 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287789+0000", 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3829999999999997e-06 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287795+0000", 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5890000000000002e-06 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334614+0000", 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046819109999999997 2024-09-13T09:28:06.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334690+0000", 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.64e-05 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334761+0000", 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0642999999999994e-05 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171298 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30777344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287805+0000", 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13483057900000001, 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047123483000000001, 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171298 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.466 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287805+0000", 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287805+0000", 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287804+0000", 2024-09-13T09:28:06.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287807+0000", 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9490000000000001e-06 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287813+0000", 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1619999999999996e-06 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287820+0000", 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3940000000000001e-06 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334786+0000", 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046966172 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334862+0000", 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6144999999999997e-05 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334928+0000", 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6169999999999998e-05 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171299 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30797824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287828+0000", 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13480767799999999, 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047264088000000003, 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.470 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171299 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287828+0000", 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287828+0000", 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287824+0000", 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999962 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287830+0000", 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3169999999999998e-06 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287837+0000", 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4810000000000003e-06 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287842+0000", 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.899e-06 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.334952+0000", 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.04711042 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335030+0000", 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8002000000000003e-05 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335092+0000", 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1673000000000004e-05 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171300 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30818304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.474 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287858+0000", 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13477779400000001, 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047406701000000002, 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171300 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287858+0000", 2024-09-13T09:28:06.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287858+0000", 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287857+0000", 2024-09-13T09:28:06.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287865+0000", 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5140000000000004e-06 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287885+0000", 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287900+0000", 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4615e-05 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335115+0000", 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047215677999999997 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335193+0000", 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.771e-05 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335264+0000", 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1279000000000007e-05 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171301 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30838784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.479 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287912+0000", 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13472315700000001, 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047507686, 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171301 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287912+0000", 2024-09-13T09:28:06.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287912+0000", 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287911+0000", 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287916+0000", 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1930000000000003e-06 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287924+0000", 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7250000000000003e-06 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287930+0000", 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335289+0000", 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047359152000000002 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335363+0000", 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4128999999999994e-05 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335420+0000", 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6960999999999999e-05 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.483 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171302 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30859264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.287934+0000", 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.134701139, 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047658264999999998, 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171302 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.484 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287934+0000", 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287934+0000", 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287934+0000", 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287937+0000", 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8310000000000001e-06 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287943+0000", 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287948+0000", 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335444+0000", 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047496570000000002 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335520+0000", 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.6107999999999997e-05 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335592+0000", 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2181999999999994e-05 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171303 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 30879744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.288186+0000", 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13444932400000001, 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047573556000000003, 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.488 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171303 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.288186+0000", 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.288186+0000", 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.287952+0000", 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9997663 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.288190+0000", 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023812599999999999 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.288197+0000", 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0110000000000003e-06 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.288202+0000", 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0660000000000003e-06 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335617+0000", 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.047414540999999998 2024-09-13T09:28:06.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335694+0000", 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7025000000000002e-05 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335760+0000", 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5542000000000004e-05 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171322 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 31268864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.292367+0000", 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.130267942, 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046216999000000002, 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171322 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292367+0000", 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292367+0000", 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292367+0000", 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292370+0000", 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.864e-06 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292382+0000", 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2431e-05 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292387+0000", 2024-09-13T09:28:06.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2279999999999998e-06 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338479+0000", 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046091877000000003 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338542+0000", 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2671999999999999e-05 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338584+0000", 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2586999999999997e-05 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171323 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 31289344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.292392+0000", 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13024336, 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046305263999999999, 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171323 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292392+0000", 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292392+0000", 2024-09-13T09:28:06.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292391+0000", 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292394+0000", 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.751e-06 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292400+0000", 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292404+0000", 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7840000000000001e-06 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338613+0000", 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046209193000000003 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338654+0000", 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0951999999999997e-05 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338697+0000", 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2987e-05 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171324 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 31309824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.292420+0000", 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13021513000000001, 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046385757, 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171324 2024-09-13T09:28:06.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292420+0000", 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292420+0000", 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292420+0000", 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292427+0000", 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3690000000000001e-06 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292436+0000", 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1400000000000006e-06 2024-09-13T09:28:06.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292442+0000", 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4349999999999996e-06 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338724+0000", 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046282052999999997 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338765+0000", 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0785000000000003e-05 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338806+0000", 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1375999999999998e-05 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.506 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171325 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 31330304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.292453+0000", 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13018275700000001, 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046469886000000002, 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171325 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292453+0000", 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292453+0000", 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292452+0000", 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292455+0000", 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6469999999999998e-06 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292461+0000", 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1020000000000002e-06 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292466+0000", 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8609999999999997e-06 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338836+0000", 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046369951999999999 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338881+0000", 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5166000000000001e-05 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338922+0000", 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1529999999999997e-05 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171326 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 31350784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.292475+0000", 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.130160567, 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046562704000000003, 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171326 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292475+0000", 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292475+0000", 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292474+0000", 2024-09-13T09:28:06.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292477+0000", 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7240000000000001e-06 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292483+0000", 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9320000000000001e-06 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292488+0000", 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2789999999999998e-06 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338949+0000", 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046460954999999998 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.338994+0000", 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4768999999999998e-05 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.339037+0000", 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3414999999999997e-05 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171327 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 31371264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.292633+0000", 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.13000272199999999, 2024-09-13T09:28:06.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046527425999999997, 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171327 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292633+0000", 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292633+0000", 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292492+0000", 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9998593 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292637+0000", 2024-09-13T09:28:06.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014454800000000001 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292646+0000", 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5729999999999997e-06 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.292661+0000", 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4302e-05 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:06.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.339066+0000", 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.046405637999999999 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.339117+0000", 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1161e-05 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.339160+0000", 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2657999999999997e-05 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.519 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:06.520 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:06.520 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:06.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:06.522 INFO:teuthology.orchestra.run.smithi063.stdout:5928 trunc from 0xa13883d to 0x138f8a0 2024-09-13T09:28:06.522 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:06.516+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:06.571 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:28:06.571 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:28:06.571 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [1, 2, 3, 5, 6, 7, 0, 4] 2024-09-13T09:28:06.571 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-13T09:28:06.621 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:06.618+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:06.658 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:06.658 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:06.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:06.672 INFO:teuthology.orchestra.run.smithi063.stdout:5929 punch from 0x46835d to 0x46d84e, (0x54f1 bytes) 2024-09-13T09:28:06.697 INFO:teuthology.orchestra.run.smithi063.stdout:5930 write 0xb61e9ea thru 0xb62bdcc (0xd3e3 bytes) 2024-09-13T09:28:06.723 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:06.716+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:06.734 INFO:teuthology.orchestra.run.smithi063.stdout:5931 trunc from 0xb62bdcd to 0xec331d 2024-09-13T09:28:06.760 INFO:teuthology.orchestra.run.smithi063.stdout:5933 write 0x24089b thru 0x240ac0 (0x226 bytes) 2024-09-13T09:28:06.782 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:06.783 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:06.783 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:06.783 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:06.783 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:06.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:06.798 INFO:teuthology.orchestra.run.smithi063.stdout:5934 trunc from 0xec331d to 0x985b23d 2024-09-13T09:28:06.809 INFO:teuthology.orchestra.run.smithi063.stdout:5935 trunc from 0x985b23d to 0xb5458aa 2024-09-13T09:28:06.819 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:06.820 INFO:teuthology.orchestra.run.smithi063.stdout:5937 write 0x28b004 thru 0x293210 (0x820d bytes) 2024-09-13T09:28:06.824 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:06.818+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:06.921 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225316480999993, 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.922 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:28:06.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225293966999999, 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:28:06.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:28:06.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:06.931 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225278130000007, 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:06.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:28:06.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225081185999997, 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:28:06.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:28:06.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.940 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225056862000002, 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:06.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:28:06.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:28:06.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225040402000005, 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.946 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:28:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.225024380999997, 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:28:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:28:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:28:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.224974048000007, 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:28:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:28:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.224870472999996, 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:28:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215492677, 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:28:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:28:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215474013000005, 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:28:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:28:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215457522999998, 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:28:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215441737999996, 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:28:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215425616999994, 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:28:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215409711999996, 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215393242999994, 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215371391000005, 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:28:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:28:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215354575999996, 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:28:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215338278999994, 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:28:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.215322678000007, 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:28:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.019 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.020 INFO:teuthology.orchestra.run.smithi063.stdout:5938 write 0x4c6c024 thru 0x4c73651 (0x762e bytes) 2024-09-13T09:28:07.020 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:28:06.992+0000 7f00df3a9700 -1 mon.a@0(leader).osd e131 definitely_dead 0 2024-09-13T09:28:07.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:07.021 INFO:teuthology.orchestra.run.smithi063.stdout:5939 trunc from 0xb5458aa to 0x3bbc6cc 2024-09-13T09:28:07.021 INFO:teuthology.orchestra.run.smithi063.stdout:5940 read 0xf61259 thru 0xf6c133 (0xaedb bytes) 2024-09-13T09:28:07.022 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:07.022 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:07.016+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:07.036 INFO:teuthology.orchestra.run.smithi063.stdout:5943 trunc from 0x3bbc6cc to 0xde2d0ce 2024-09-13T09:28:07.054 INFO:teuthology.orchestra.run.smithi063.stdout:5944 trunc from 0xde2d0ce to 0xc22082a 2024-09-13T09:28:07.068 INFO:teuthology.orchestra.run.smithi063.stdout:5945 punch from 0x2c90686 to 0x2c964d4, (0x5e4e bytes) 2024-09-13T09:28:07.068 INFO:teuthology.orchestra.run.smithi063.stdout:5947 writesame 0xb674e9d thru 0xb67b9a1 data_size 0x1(0x6b05 bytes) 2024-09-13T09:28:07.094 INFO:teuthology.orchestra.run.smithi063.stdout:5948 read 0x3e391c8 thru 0x3e42027 (0x8e60 bytes) 2024-09-13T09:28:07.094 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:07.094 INFO:teuthology.orchestra.run.smithi063.stdout:5951 read 0x4f447d4 thru 0x4f4eac1 (0xa2ee bytes) 2024-09-13T09:28:07.094 INFO:teuthology.orchestra.run.smithi063.stdout:5952 write 0xa7b2ad3 thru 0xa7ba030 (0x755e bytes) 2024-09-13T09:28:07.112 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:07.112 INFO:teuthology.orchestra.run.smithi063.stdout:5954 read 0x3a125cd thru 0x3a18ac8 (0x64fc bytes) 2024-09-13T09:28:07.112 INFO:teuthology.orchestra.run.smithi063.stdout:5955 read 0x142c890 thru 0x14329fd (0x616e bytes) 2024-09-13T09:28:07.113 INFO:teuthology.orchestra.run.smithi063.stdout:5956 trunc from 0xc22082a to 0xba83a60 2024-09-13T09:28:07.120 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:07.114+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:07.123 INFO:teuthology.orchestra.run.smithi063.stdout:5957 read 0x751e53c thru 0x752211c (0x3be1 bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5958 punch from 0x93bd57 to 0x9416a1, (0x594a bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5960 punch from 0x8a2a6f2 to 0x8a36903, (0xc211 bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5961 read 0x5269d2d thru 0x526d525 (0x37f9 bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5962 read 0x9e8349d thru 0x9e86efd (0x3a61 bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5964 read 0x913c4bb thru 0x9142c6e (0x67b4 bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5966 read 0x9447a48 thru 0x944f764 (0x7d1d bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5967 read 0xa2c1a0 thru 0xa2f7fa (0x365b bytes) 2024-09-13T09:28:07.124 INFO:teuthology.orchestra.run.smithi063.stdout:5968 read 0x9ed8f5e thru 0x9ee5c89 (0xcd2c bytes) 2024-09-13T09:28:07.125 INFO:teuthology.orchestra.run.smithi063.stdout:5969 read 0x9d874f0 thru 0x9d8f9ab (0x84bc bytes) 2024-09-13T09:28:07.145 INFO:teuthology.orchestra.run.smithi063.stdout:5970 trunc from 0xba83a60 to 0x1eb3b1f 2024-09-13T09:28:07.170 INFO:teuthology.orchestra.run.smithi063.stdout:5971 punch from 0x9079e6 to 0x907cd0, (0x2ea bytes) 2024-09-13T09:28:07.170 INFO:teuthology.orchestra.run.smithi063.stdout:5972 read 0x9651d6 thru 0x96dff7 (0x8e22 bytes) 2024-09-13T09:28:07.192 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:07.193 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:07.193 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:07.193 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:07.249 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:07.246+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:07.273 INFO:teuthology.orchestra.run.smithi063.stdout:5973 trunc from 0x1eb3b1f to 0xc0f20fb 2024-09-13T09:28:07.322 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:07.318+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:07.323 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:07.323 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:07.339 INFO:teuthology.orchestra.run.smithi063.stdout:5975 write 0x9970289 thru 0x997495e (0x46d6 bytes) 2024-09-13T09:28:07.360 INFO:teuthology.orchestra.run.smithi063.stdout:5977 read 0x7857891 thru 0x7865af9 (0xe269 bytes) 2024-09-13T09:28:07.360 INFO:teuthology.orchestra.run.smithi063.stdout:5979 read 0xa48f8ee thru 0xa496e5f (0x7572 bytes) 2024-09-13T09:28:07.368 INFO:teuthology.orchestra.run.smithi063.stdout:5980 write 0xc301c6b thru 0xc302c80 (0x1016 bytes) 2024-09-13T09:28:07.382 INFO:teuthology.orchestra.run.smithi063.stdout:5983 read 0x94e88c8 thru 0x94f7f54 (0xf68d bytes) 2024-09-13T09:28:07.383 INFO:teuthology.orchestra.run.smithi063.stdout:5984 read 0x5b4afb9 thru 0x5b4f8c0 (0x4908 bytes) 2024-09-13T09:28:07.383 INFO:teuthology.orchestra.run.smithi063.stdout:5986 write 0xbdc4746 thru 0xbdcb284 (0x6b3f bytes) 2024-09-13T09:28:07.405 INFO:teuthology.orchestra.run.smithi063.stdout:5987 write 0x9f454f7 thru 0x9f48933 (0x343d bytes) 2024-09-13T09:28:07.419 INFO:teuthology.orchestra.run.smithi063.stdout:5989 punch from 0x893fe7f to 0x89476b2, (0x7833 bytes) 2024-09-13T09:28:07.420 INFO:teuthology.orchestra.run.smithi063.stdout:5990 writesame 0xb6290e thru 0xb6554c data_size 0x1(0x2c3f bytes) 2024-09-13T09:28:07.423 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:07.416+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:07.429 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:07.429 INFO:teuthology.orchestra.run.smithi063.stdout:5992 punch from 0xba4297e to 0xba4b160, (0x87e2 bytes) 2024-09-13T09:28:07.429 INFO:teuthology.orchestra.run.smithi063.stdout:5995 read 0xc14ff1d thru 0xc156e8c (0x6f70 bytes) 2024-09-13T09:28:07.429 INFO:teuthology.orchestra.run.smithi063.stdout:5996 punch from 0x5ccda60 to 0x5cdc8f6, (0xee96 bytes) 2024-09-13T09:28:07.429 INFO:teuthology.orchestra.run.smithi063.stdout:compare and write disabled 2024-09-13T09:28:07.429 INFO:teuthology.orchestra.run.smithi063.stdout:5999 punch from 0x6d561d to 0x6d86d6, (0x30b9 bytes) 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.364525189999995, 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:07.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:07.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.141284599000002, 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:07.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.614071393000003, 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:07.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3548244440000001, 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:07.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3548078889999999, 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:07.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.476 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3547928250000001, 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:07.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168642 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10797056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092533+0000", 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3540797549999999, 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049748936000000001, 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168642 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092533+0000", 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092533+0000", 2024-09-13T09:28:07.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092532+0000", 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092535+0000", 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092540+0000", 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092545+0000", 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6789999999999996e-06 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142155+0000", 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049610083999999999 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142222+0000", 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6905999999999999e-05 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142282+0000", 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9681e-05 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.354061062, 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:07.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.354037156, 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:07.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:07.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.353345156, 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.35333112, 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:07.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3532127430000001, 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:07.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3530196110000001, 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:07.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3490844909999999, 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:07.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:07.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.349075805, 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:07.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3490664450000001, 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:07.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:07.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.349057666, 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.349049087, 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3490364029999999, 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:07.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3490275110000001, 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:07.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.544 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.546 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:07.546 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:07.540+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:07.547 INFO:teuthology.orchestra.run.smithi063.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-13T09:28:07.600 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:28:07.608 INFO:teuthology.orchestra.run.smithi063.stderr:marked out osd.4. 2024-09-13T09:28:07.663 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:07.663 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:07.663 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:07.663 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.664 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:07.660+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:07.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:07.749 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:07.744+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:07.775 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:07.776 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:07.776 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:07.776 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:07.776 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:07.850 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:07.844+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:07.896 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.468603009, 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.901 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4685879530000001, 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:07.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.468578545, 2024-09-13T09:28:07.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:07.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:07.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.468564252, 2024-09-13T09:28:07.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4684785460000001, 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.916 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4681222970000001, 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:07.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.468075461, 2024-09-13T09:28:07.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4680364749999999, 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.929 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:07.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:07.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4680163310000001, 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:07.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467997263, 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:07.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:07.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.943 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4679551740000001, 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:07.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:07.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467841672, 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.949 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:07.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467823493, 2024-09-13T09:28:07.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:07.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:07.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467813748, 2024-09-13T09:28:07.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:07.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467803561, 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:07.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:07.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4677106950000001, 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:07.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.467607283, 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:07.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.4672854049999999, 2024-09-13T09:28:07.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:07.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:07.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:07.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.466869464, 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:07.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.466615754, 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:07.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:07.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:07.995 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:07.999 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:07.996+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:08.000 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:08.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:08.098 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:08.092+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:08.111 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:08.112 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:08.112 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:08.112 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:08.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:08.166 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:08.189 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:08.199 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:08.194+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:08.211 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:08.226 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:08.226 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:08.226 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:08.227 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:08.227 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:08.232 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:08.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:08.254 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:08.275 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:08.297 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:08.300 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:08.294+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:08.321 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:08.323 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:08.324 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:08.342 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:08.342 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:08.352 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:08.401 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:08.394+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:08.467 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:08.467 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:08.477 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:08.502 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:08.496+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:08.587 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:08.587 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:08.597 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:08.615 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:28:08.615 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:08.608+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:08.703 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:08.696+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:08.714 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:08.714 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:08.724 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:08.804 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:08.799+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:08.832 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:08.833 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:08.842 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:08.905 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:08.899+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:08.994 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:08.995 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:09.006 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:08.999+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:09.107 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:09.102+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:09.113 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.113 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.123 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:09.208 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:09.202+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:09.237 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.237 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:09.309 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:09.302+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.356 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.366 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:09.410 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:09.403+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:09.471 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.472 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.481 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:09.511 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:09.506+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:09.612 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:09.606+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:09.657 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.657 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:09.712 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:09.706+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.799 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.810 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:09.813 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:09.806+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:09.913 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:09.907+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:09.937 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:09.937 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:09.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:10.015 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:10.010+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:10.082 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:10.082 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:10.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:10.116 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:10.111+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:10.204 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:10.205 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:10.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:10.219 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:10.211+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:10.318 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:10.311+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:10.337 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:10.340 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:10.340 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:10.354 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:10.419 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:10.414+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:10.470 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:10.470 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:10.481 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:10.496 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:10.520 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:10.514+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:10.521 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:10.544 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:10.565 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:10.587 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:10.600 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:10.600 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:10.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:10.611 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:10.621 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:10.615+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:10.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:28:10.639 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:10.666 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:10.668 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:10.669 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:10.723 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:10.715+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:10.769 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:10.769 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.684340782999996, 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:10.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:10.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.774 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 59.461100192000004, 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:10.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:10.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:10.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.933886985999997, 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:10.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:10.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:10.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6746400369999996, 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6746234820000003, 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:10.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:10.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.674608418, 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:10.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168642 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10797056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092533+0000", 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6738953480000003, 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049748936000000001, 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168642 2024-09-13T09:28:10.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092533+0000", 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092533+0000", 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092532+0000", 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092535+0000", 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.044e-06 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092540+0000", 2024-09-13T09:28:10.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092545+0000", 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6789999999999996e-06 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142155+0000", 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049610083999999999 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142222+0000", 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6905999999999999e-05 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142282+0000", 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9681e-05 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6738766549999999, 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6738527489999999, 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:10.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:10.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.673160749, 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:10.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6731467129999995, 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6730283359999998, 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:10.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:10.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6728352040000001, 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:10.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:10.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6689000839999997, 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.839 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6688913979999995, 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:10.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6688820379999996, 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6688732589999997, 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:10.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6688646800000004, 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:10.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6688519959999999, 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:10.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:10.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:10.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6688431040000005, 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:10.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:10.870 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:10.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:10.872 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:10.863+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:10.973 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:10.970+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:10.996 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:10.997 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:10.997 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:11.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:11.074 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:11.070+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:11.117 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:11.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:11.175 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:11.167+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:11.275 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:11.270+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:11.301 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8733150849999998, 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:11.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8733000290000001, 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:11.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:11.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8732906209999998, 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8732763280000002, 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:11.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:11.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8731906220000001, 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:11.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8728343729999999, 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:11.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8727875369999998, 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:11.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:11.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8727485509999999, 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:11.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:11.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8727284070000003, 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:11.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.872709339, 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:11.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8726672500000001, 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:11.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:11.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:11.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8725537479999996, 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8725355690000001, 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8725258240000002, 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:11.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:11.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8725156370000002, 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.365 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.369 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8724227710000001, 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:11.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8723193590000005, 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:11.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:11.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8719974810000002, 2024-09-13T09:28:11.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:11.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:11.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8715815400000002, 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:11.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:11.387 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8713278300000002, 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:11.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:11.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:11.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.392 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:11.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:11.394 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:11.387+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:11.494 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:11.487+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:11.512 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:11.513 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:11.513 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:11.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:11.595 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:11.590+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:11.624 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:11.624 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:11.624 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:11.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:11.653 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:28:11.712 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:11.707+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:11.753 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:11.753 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 148.78839550800001, 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:28:11.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:28:11.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:11.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 85.675175764000002, 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:28:11.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:28:11.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 85.674777749, 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:28:11.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:28:11.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:11.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999670872999999, 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:28:11.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999655941, 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:28:11.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999622834999997, 2024-09-13T09:28:11.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:28:11.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999606541999995, 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999590427000001, 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:28:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:28:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999559386000001, 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:28:11.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.999527763000003, 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:28:11.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.798 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986579702, 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:11.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:28:11.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986562645999996, 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:28:11.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986546477999994, 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:11.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:28:11.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986332816000001, 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:28:11.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:11.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:28:11.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986313504999998, 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.821 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986297112000003, 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:28:11.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:28:11.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:28:11.826 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986281450000007, 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.827 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.986255095999994, 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.832 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:11.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.457742369000002, 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:28:11.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:28:11.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:28:11.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:28:11.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.345828253999997, 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:11.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:28:11.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:28:11.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:11.848 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:11.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:11.850 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:11.843+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:11.951 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:11.943+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:11.968 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:11.969 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:11.969 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:11.969 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:11.979 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:12.052 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:12.047+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:12.134 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:12.134 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:12.134 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:12.134 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:12.135 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:12.144 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:12.152 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:12.146+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:12.253 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:12.247+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:12.255 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:12.255 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:12.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:12.255 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:12.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.515227378999995, 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:12.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:12.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.262 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2730546279999997, 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:12.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:12.266 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.273031413, 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.267 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:12.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:12.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2689030959999998, 2024-09-13T09:28:12.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:12.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:12.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2688505360000004, 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:12.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:12.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2688138410000001, 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.280 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:12.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:12.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:12.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:12.286 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2687942110000003, 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2687720320000002, 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.291 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:12.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2650162490000003, 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:12.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2649909150000003, 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.300 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:12.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2649730779999997, 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:12.304 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:12.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:12.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2649606929999999, 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:12.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2649319319999996, 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:12.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:12.318 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2607797210000005, 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:12.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:12.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2607514330000003, 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.323 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:12.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2607252989999997, 2024-09-13T09:28:12.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.331 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2607006619999996, 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:12.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:12.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:12.336 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2606180880000002, 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:12.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2605751850000004, 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.341 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:12.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:12.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.2569494629999998, 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:12.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:12.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.350 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.350 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:12.351 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:12.354 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:12.350+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:12.454 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:12.450+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:12.470 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:12.470 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:12.470 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:12.470 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:12.480 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:12.556 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:12.550+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:12.590 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:12.590 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:12.591 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:12.591 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:12.591 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:12.600 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:12.622 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:28:12.622 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:28:12.622 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [1, 2, 3, 5, 6, 7, 0] 2024-09-13T09:28:12.622 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-13T09:28:12.657 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:12.650+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:12.683 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:12.715 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.151032204000003, 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.716 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:12.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:12.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 58.563373824999999, 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:12.721 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:12.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:12.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.726 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 6.4330816999999998, 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:12.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.8488712490000001, 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:12.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.735 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4475376170000001, 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.736 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:12.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4475196090000004, 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.740 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:12.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4475022700000002, 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:12.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4474850129999997, 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.749 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:12.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.447467477, 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:12.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4474504069999998, 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.758 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:12.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4474367480000003, 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.763 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:12.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:12.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4474270750000002, 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:12.767 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:12.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:12.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4474125300000003, 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:12.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4473958629999997, 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.776 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:12.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4473787720000004, 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:12.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4455231959999999, 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:12.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4455070880000003, 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4454927240000002, 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:12.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.445478896, 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:12.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 5.4454611960000001, 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:12.803 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:12.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:12.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.807 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:12.808 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:12.808 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:12.809 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:12.810 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:12.802+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:12.819 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:12.841 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:12.870 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:12.890 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:12.910 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:12.903+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:12.920 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:12.933 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:12.934 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:12.942 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:12.943 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:12.944 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:28:12.935+0000 7f00df3a9700 -1 mon.a@0(leader).osd e137 definitely_dead 0 2024-09-13T09:28:12.964 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:12.987 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:12.989 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:12.990 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:13.011 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:13.006+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:13.045 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.046 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.055 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:13.112 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:13.103+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:13.162 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.162 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.172 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:13.213 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:13.210+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:13.283 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.283 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.293 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:13.314 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:13.310+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:13.402 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.402 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.412 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:13.414 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:13.410+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:13.517 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:13.514+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:13.521 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.522 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:13.618 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:13.611+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:13.644 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.645 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:13.668 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:28:13.685 INFO:teuthology.orchestra.run.smithi063.stderr:marked out osd.2. 2024-09-13T09:28:13.718 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:13.714+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:13.758 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.758 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:13.819 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:13.814+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:13.873 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.874 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:13.920 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:13.915+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:13.984 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:13.984 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:13.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:14.038 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:14.034+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:14.102 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.102 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:14.122 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:14.118+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:14.220 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.220 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.222 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:14.218+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:14.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:14.323 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:14.318+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:14.384 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.384 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:14.441 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:14.438+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:14.501 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.501 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:14.525 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:14.523+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:14.614 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.614 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:14.625 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:14.619+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:14.691 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:28:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.732 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:14.744 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:14.739+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:14.827 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:14.822+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:14.845 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:14.845 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:14.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:14.928 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:14.922+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:15.001 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:15.009 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:15.009 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:15.019 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:15.029 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:15.026+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:15.130 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:15.126+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:15.146 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:15.146 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:15.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:15.166 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:15.186 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:15.204 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:15.222 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:15.231 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:15.223+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:15.245 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:15.258 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:15.258 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:15.258 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:15.258 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:15.258 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:15.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:15.269 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:15.292 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:15.314 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:15.317 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:15.317 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:15.317 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:15.317 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:15.318 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:15.318 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:15.318 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:15.319 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:15.376 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:15.371+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:15.381 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.639871072999995, 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.382 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:15.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:15.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:15.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:15.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:15.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.389 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3976983220000001, 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:15.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3976751069999995, 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:15.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:15.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:15.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:15.399 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3935467900000003, 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.400 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:15.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:15.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3934942299999999, 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:15.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3934575349999996, 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.409 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:15.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:15.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3934379050000008, 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:15.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3934157260000006, 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:15.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.423 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3896599429999998, 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:15.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:15.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:15.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:15.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:15.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.429 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.429 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.429 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3896346089999998, 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.431 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:15.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.431 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:15.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:15.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:15.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.437 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.438 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:15.438 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3896167720000001, 2024-09-13T09:28:15.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:15.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.439 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.439 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:15.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.440 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:15.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:15.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:15.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:15.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:15.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:15.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:15.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:15.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:15.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:15.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:15.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:15.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:15.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:15.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:15.448 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.448 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.448 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.449 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.449 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:15.449 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3896043870000003, 2024-09-13T09:28:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.451 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:15.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.451 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:15.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:15.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:15.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:15.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:15.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:15.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:15.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:15.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:15.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:15.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:15.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:15.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:15.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:15.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:15.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.459 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.460 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.460 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:15.460 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3895756259999992, 2024-09-13T09:28:15.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:15.461 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.461 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.462 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.462 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:15.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.462 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:15.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:15.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:15.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:15.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:15.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:15.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.385423415, 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:15.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3853951270000007, 2024-09-13T09:28:15.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3853689930000002, 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.479 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:15.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3853443559999992, 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.484 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:15.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.489 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3852617820000006, 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.494 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.385218879, 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.495 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:15.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.3815931569999993, 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:15.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:15.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.504 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:15.506 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:15.503+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:15.506 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:15.607 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:15.606+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:15.623 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:15.623 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:15.634 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:15.708 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:15.703+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:15.756 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:15.757 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:15.757 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:15.757 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:15.757 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:15.769 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:15.808 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:15.802+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:15.910 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:15.906+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:15.932 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:15.933 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:15.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:15.933 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:15.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.933 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:15.933 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:15.934 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 94.366600743000006, 2024-09-13T09:28:15.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:15.934 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.934 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:15.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:15.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:15.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.778942364000002, 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:15.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:15.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:15.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:15.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6486502390000002, 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:15.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:15.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:15.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.0644397879999996, 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.952 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6631061559999996, 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.957 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:15.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6630881479999999, 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.962 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:15.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.967 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6630708090000006, 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:15.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:15.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6630535520000009, 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:15.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:15.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:15.976 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6630360159999995, 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:15.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:15.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6630189459999993, 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:15.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:15.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6630052870000007, 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:15.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.990 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6629956139999997, 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:15.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:15.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:15.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6629810690000006, 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:15.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:15.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:15.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:15.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.001 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6629644020000001, 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:16.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6629473109999999, 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.006 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:16.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.010 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6610917349999994, 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:16.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6610756270000007, 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.015 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6610612630000006, 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:16.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6610474350000004, 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.024 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 8.6610297349999996, 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:16.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.033 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:16.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:16.035 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:16.027+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:16.135 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:16.130+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:16.154 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:16.154 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:16.154 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:16.154 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:16.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:16.236 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:16.230+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:16.291 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:16.291 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:16.292 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:16.292 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:16.292 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:16.303 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:16.337 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:16.331+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840377773, 2024-09-13T09:28:16.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:16.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:16.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:16.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:16.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:16.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:16.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:16.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:16.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:16.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.427 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.428 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.428 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:16.428 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840365536, 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:16.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.433 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840350139000002, 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:16.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840335264, 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:16.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:16.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.442 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840318055000001, 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:16.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:16.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.447 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.8402976, 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:16.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840279519999999, 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.452 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:16.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:16.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:16.456 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840263144000001, 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:16.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:16.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840247793, 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.461 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:16.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840231089, 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:16.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:16.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:16.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840213006999999, 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.471 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:16.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.475 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840196261999999, 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.476 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:16.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.480 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.840151602999999, 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:16.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.837202163000001, 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.485 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:16.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.490 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.490 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:16.490 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.837181624999999, 2024-09-13T09:28:16.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:16.490 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.491 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:16.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:16.495 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.837159010000001, 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.496 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:16.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:16.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:16.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.502 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.502 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:16.502 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.837140141999999, 2024-09-13T09:28:16.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:16.502 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.502 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:16.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:16.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:16.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:16.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.837105548, 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.832826643000001, 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.832762671000001, 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.524 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:16.525 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:16.526 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:16.522+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:16.626 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:16.619+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:16.658 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:16.658 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:16.659 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:16.659 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:16.669 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:16.703 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:28:16.727 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:16.722+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:16.786 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:16.786 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:16.786 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:16.787 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:16.787 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:16.798 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:16.827 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:16.819+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:16.910 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:16.910 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:16.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:16.910 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:16.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.910 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 15.347281102, 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:16.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:16.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.915 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.916 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.916 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:16.916 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.7189135279999999, 2024-09-13T09:28:16.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:16.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6773349339999992, 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:16.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:16.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6759197550000007, 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:16.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:16.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.930 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6759011019999992, 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:16.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:16.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6758285540000006, 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:16.935 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:16.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:16.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:16.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6755917789999994, 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.940 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:16.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6755489479999994, 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:16.945 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:16.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:16.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.949 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6755252859999992, 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.950 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.952 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:16.953 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6753152549999992, 2024-09-13T09:28:16.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:16.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:16.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:16.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6752883839999999, 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:16.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:16.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:16.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6752704899999991, 2024-09-13T09:28:16.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:16.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6752542429999995, 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:16.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:16.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:16.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.675156179, 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6749229440000004, 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6748283369999992, 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6748028119999994, 2024-09-13T09:28:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.674779633, 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6745056100000006, 2024-09-13T09:28:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 9.6743214589999997, 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:17.009 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:17.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:17.011 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:17.003+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:17.111 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:17.106+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:17.137 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:17.137 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:17.137 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:17.137 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:17.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:17.212 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:17.207+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:17.258 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:17.258 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:17.258 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:17.258 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:17.258 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:17.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:17.317 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:17.314+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:17.329 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:17.375 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85197 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 684032~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678749261999997, 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065997439000000005, 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85197 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693472+0000", 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693471+0000", 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693478+0000", 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:28:17.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693485+0000", 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693490+0000", 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1800000000000004e-06 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759229+0000", 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065739639000000002 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759316+0000", 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6458999999999993e-05 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759469+0000", 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153289 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85198 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 699392~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678726748000003, 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066176254000000004, 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85198 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693494+0000", 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693496+0000", 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693501+0000", 2024-09-13T09:28:17.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3029999999999999e-06 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693506+0000", 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759494+0000", 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065987828999999998 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759562+0000", 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8359000000000001e-05 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759670+0000", 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108139 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85199 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 652288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.385 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678710910999996, 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066347825999999999, 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85199 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:17.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693510+0000", 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693509+0000", 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693512+0000", 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693519+0000", 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3529999999999997e-06 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693525+0000", 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6969999999999998e-06 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759695+0000", 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066170108000000005 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759756+0000", 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0507000000000002e-05 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759858+0000", 2024-09-13T09:28:17.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010226999999999999 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85200 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 714752~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678513967000001, 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066273440000000003, 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85200 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693707+0000", 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693529+0000", 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998217 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693711+0000", 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000181962 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693719+0000", 2024-09-13T09:28:17.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3939999999999996e-06 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693726+0000", 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.759e-06 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759882+0000", 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066156380000000001 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759929+0000", 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6996999999999997e-05 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.759980+0000", 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1081000000000001e-05 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85201 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 667648~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.394 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678489643000006, 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402849999999999, 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85201 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:17.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693731+0000", 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693734+0000", 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6879999999999999e-06 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693739+0000", 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3000000000000001e-06 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693744+0000", 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9200000000000003e-06 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760003+0000", 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066259314 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760066+0000", 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2848999999999994e-05 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760134+0000", 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8215999999999999e-05 2024-09-13T09:28:17.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85202 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 730112~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678473182999994, 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066543912999999996, 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85202 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693748+0000", 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693747+0000", 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693750+0000", 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693755+0000", 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4199999999999998e-06 2024-09-13T09:28:17.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693760+0000", 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9699999999999998e-06 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760162+0000", 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066402181000000005 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760223+0000", 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1029999999999997e-05 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760292+0000", 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8384000000000001e-05 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85203 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 683008~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:17.403 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678457162000001, 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066700252000000002, 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85203 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693764+0000", 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693763+0000", 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693766+0000", 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693771+0000", 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.259e-06 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693776+0000", 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6670000000000004e-06 2024-09-13T09:28:17.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760317+0000", 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066541672999999996 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760377+0000", 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0065999999999997e-05 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760464+0000", 2024-09-13T09:28:17.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6698000000000002e-05 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85204 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 745472~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678406828999996, 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066823726, 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.408 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85204 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693814+0000", 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693779+0000", 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999647 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693817+0000", 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7897e-05 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693824+0000", 2024-09-13T09:28:17.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1550000000000001e-06 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693830+0000", 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7760000000000003e-06 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760489+0000", 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066659606999999996 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760550+0000", 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0270999999999999e-05 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760638+0000", 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8078999999999995e-05 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85205 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 760832~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.678303253999999, 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066903957, 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85205 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:17.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693918+0000", 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693835+0000", 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999918 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.693921+0000", 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5721000000000001e-05 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.697286+0000", 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033650030000000001 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.698742+0000", 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001455855 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760664+0000", 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061921832000000003 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760723+0000", 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9231999999999998e-05 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.760822+0000", 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8548000000000006e-05 2024-09-13T09:28:17.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85237 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 882688~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668925458000004, 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065927680000000002, 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85237 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703295+0000", 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703298+0000", 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703304+0000", 2024-09-13T09:28:17.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.528e-06 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703310+0000", 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2079999999999999e-06 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.768945+0000", 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065635355000000006 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769005+0000", 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0278999999999997e-05 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769223+0000", 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00021787099999999999 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85239 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 898048~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668906793999994, 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066078340999999999, 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85239 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703314+0000", 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703316+0000", 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2919999999999998e-06 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703322+0000", 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703326+0000", 2024-09-13T09:28:17.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7740000000000004e-06 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769249+0000", 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.065922562000000004 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769318+0000", 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8703999999999998e-05 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769392+0000", 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4747999999999997e-05 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85241 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 913408~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668890304000001, 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066223211000000004, 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85241 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703331+0000", 2024-09-13T09:28:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703330+0000", 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703333+0000", 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.272e-06 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703338+0000", 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703343+0000", 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5820000000000003e-06 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769417+0000", 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066074715000000006 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769485+0000", 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7908000000000001e-05 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769554+0000", 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8676000000000005e-05 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85243 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 928768~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668874518999999, 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066365959000000002, 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85243 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703346+0000", 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703348+0000", 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.379e-06 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703354+0000", 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4079999999999997e-06 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703358+0000", 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:28:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769577+0000", 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066218733000000002 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769642+0000", 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4480000000000006e-05 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769712+0000", 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0678000000000003e-05 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85245 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 944128~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668858397999998, 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066515278999999997, 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85245 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703363+0000", 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703362+0000", 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703365+0000", 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.418e-06 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703370+0000", 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3800000000000002e-06 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703375+0000", 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.549e-06 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769736+0000", 2024-09-13T09:28:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066361407999999997 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769801+0000", 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5492000000000002e-05 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769878+0000", 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6364000000000003e-05 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85247 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 959488~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668842493, 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066660129999999998, 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85247 2024-09-13T09:28:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703378+0000", 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703381+0000", 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3740000000000001e-06 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703386+0000", 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703390+0000", 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4580000000000003e-06 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769904+0000", 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066513526000000003 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.769969+0000", 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5074999999999996e-05 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770039+0000", 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9923000000000001e-05 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85249 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 974848~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668826023999998, 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066795779, 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85249 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703395+0000", 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703397+0000", 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703402+0000", 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4729999999999999e-06 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703406+0000", 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1300000000000003e-06 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770061+0000", 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066654802999999999 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770123+0000", 2024-09-13T09:28:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1766000000000005e-05 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770191+0000", 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7670999999999996e-05 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85251 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 990208~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668804171999994, 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066921709999999995, 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85251 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703417+0000", 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703416+0000", 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703419+0000", 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7640000000000001e-06 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703425+0000", 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.835e-06 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703430+0000", 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7129999999999996e-06 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770213+0000", 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066783964000000001 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770272+0000", 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8109000000000002e-05 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770338+0000", 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6876000000000001e-05 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85252 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1005568~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668787356999999, 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067061520999999999, 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85252 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703434+0000", 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703433+0000", 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703436+0000", 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703441+0000", 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3109999999999996e-06 2024-09-13T09:28:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703446+0000", 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770361+0000", 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.066915346000000001 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770423+0000", 2024-09-13T09:28:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1495000000000007e-05 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770495+0000", 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2550000000000002e-05 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85254 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1020928~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668771059999997, 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067196124999999995, 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85254 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703450+0000", 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703449+0000", 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703452+0000", 2024-09-13T09:28:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2170000000000001e-06 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703457+0000", 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2630000000000003e-06 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703461+0000", 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5240000000000002e-06 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770519+0000", 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067057137000000003 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770578+0000", 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9886000000000002e-05 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770646+0000", 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7514000000000004e-05 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:85257 3.3 3:c3205f3d:::rbd_data.111838d2a860.0000000000000005:17 [read 1036288~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.668755458999996, 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067356786000000002, 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 85257 2024-09-13T09:28:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703465+0000", 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703472+0000", 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703478+0000", 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7189999999999998e-06 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.703483+0000", 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9049999999999996e-06 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770675+0000", 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.067191901999999998 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770744+0000", 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9196000000000006e-05 2024-09-13T09:28:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.770822+0000", 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8022999999999995e-05 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:17.469 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:17.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:17.471 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:17.466+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:17.484 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:17.503 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:17.521 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:17.541 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:17.564 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:17.571 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:17.566+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:17.588 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:17.588 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:17.588 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:17.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:17.609 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:17.633 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:17.636 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:17.673 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:17.667+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:17.715 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:28:17.761 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:17.761 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:17.771 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:17.773 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:17.767+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:17.874 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:17.867+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:17.881 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:17.881 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:17.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:17.976 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:17.970+0000 7f0b79fc1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 22310) UID: 0 2024-09-13T09:28:17.996 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:17.996 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:18.077 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:18.070+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:18.114 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.114 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:18.177 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:18.174+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:18.230 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.230 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:18.278 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:18.271+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:18.347 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.347 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:18.379 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:18.374+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:18.463 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.463 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.473 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:18.480 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:18.475+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:18.579 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.579 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.581 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:18.578+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:18.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:18.683 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:18.675+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:18.697 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0] out_osds: [4, 2] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:28:18.697 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:28:18.698 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-13T09:28:18.698 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T09:28:18.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:28:18.711 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.711 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.725 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:18.783 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:18.775+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:18.797 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T09:28:18.797 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-13T09:28:18.798 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:28:18.798 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-13T09:28:18.804 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:28:18.804 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-13T09:28:18.824 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.824 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.833 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:18.884 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:18.878+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:18.938 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:18.939 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:18.949 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:18.985 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:18.979+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:19.057 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:19.057 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:19.066 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:19.086 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:19.082+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:19.187 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:19.182+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:19.219 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:19.219 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:19.229 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:19.288 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:19.283+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:19.339 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:19.339 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:19.349 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:19.392 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:19.387+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:19.462 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:19.462 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:19.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:19.516 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:19.511+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:19.591 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:19.583+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:19.624 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:19.624 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:19.634 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:19.648 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:19.692 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:19.687+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:19.731 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:28:19.746 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:19.746 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:19.763 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:19.808 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:19.803+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:19.812 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:19.851 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:19.882 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:19.892 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:19.886+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:19.914 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:19.917 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:19.917 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:19.917 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:19.917 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:19.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:19.941 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:19.964 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:19.986 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:19.987 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:19.990 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:19.991 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:19.993 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:19.987+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:20.051 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:20.052 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:20.052 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:20.052 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:20.052 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:20.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:20.104 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:20.099+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:20.199 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:20.191+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:20.214 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.652398092999999, 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.215 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:20.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 13.024030519, 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.221 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:20.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.982451924999999, 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:20.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.981036746000001, 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.229 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:20.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.233 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.981018092999999, 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:20.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:20.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980945545000001, 2024-09-13T09:28:20.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.242 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.98070877, 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980665939, 2024-09-13T09:28:20.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980642276999999, 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:20.252 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:20.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:20.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:20.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:20.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:20.257 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980432245999999, 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:20.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:20.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980405375, 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:20.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.266 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980387480999999, 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:20.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.271 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980371234, 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:20.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.98027317, 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.276 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:20.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.980039935000001, 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.281 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:20.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:20.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.979945327999999, 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:20.285 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:20.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.289 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.979919803, 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.290 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:20.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:20.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.295 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.979896624, 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:20.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:20.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:20.300 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.979622600999999, 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:20.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 12.97943845, 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:20.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:20.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:20.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:20.309 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:20.310 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:20.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:20.312 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:20.306+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:20.412 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:20.407+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:20.420 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:20.420 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:20.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:20.482 INFO:teuthology.orchestra.run.smithi099.stdout:3.13 2024-09-13T09:28:20.513 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:20.511+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:20.530 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:20.530 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:20.530 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:20.530 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:20.530 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:20.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:20.614 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:20.610+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:20.643 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.557861790000004, 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:20.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.648 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 69.334621198999997, 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:20.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:20.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:20.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.807407992999998, 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:20.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.548161044, 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.548144489, 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:20.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.548129424999999, 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:20.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.547397662, 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.680 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.547373756000001, 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:20.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.546681756, 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.54666772, 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:20.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.546549343000001, 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.698 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.546356210999999, 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:20.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:20.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:20.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:20.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542421091, 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542412405, 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.708 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:20.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:20.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542403045, 2024-09-13T09:28:20.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:20.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542394266000001, 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:20.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542385686999999, 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:20.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.725 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542373003, 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.726 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:20.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.542364110999999, 2024-09-13T09:28:20.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:20.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:182783 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.9765372350000003, 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99356371799999998, 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:20.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 182783 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:20.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663410+0000", 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663420+0000", 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0097e-05 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663440+0000", 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9559999999999999e-05 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663448+0000", 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.824e-06 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663515+0000", 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6964999999999993e-05 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663527+0000", 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2295e-05 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656700+0000", 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99317316300000003 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656806+0000", 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105889 2024-09-13T09:28:20.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656976+0000", 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016978699999999999 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:20.740 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:20.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:20.742 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:20.738+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:20.760 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:28:20.843 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:20.835+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:20.856 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:20.856 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:20.857 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:20.857 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:20.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:20.944 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:20.938+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:20.971 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:20.971 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:20.971 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:20.971 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:20.971 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:20.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:21.045 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:21.039+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:21.055 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.13 --file /var/log/ceph/exp.3.13.4 2024-09-13T09:28:21.139 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:21.139 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:21.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:21.139 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:21.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.139 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711696924, 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:21.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711681867999999, 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:21.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711672459999999, 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:21.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:21.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711658167, 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:21.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711572460999999, 2024-09-13T09:28:21.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:21.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:21.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:21.164 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711216212, 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:21.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:21.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711169376000001, 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711130389999999, 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:21.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711110246000001, 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.179 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:21.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:21.183 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711091178, 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:21.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:21.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711049088999999, 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:21.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:21.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.710935587, 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:21.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:21.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.197 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.710917408, 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:21.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.710907663, 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:21.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.206 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.710897476, 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:21.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:21.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.71080461, 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:21.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.710701198000001, 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:21.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.710379319999999, 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:21.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.709963379, 2024-09-13T09:28:21.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:21.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:21.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.709709669, 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:21.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:21.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:21.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:21.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.235 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:21.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:21.237 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:21.231+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:21.338 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:21.334+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:21.350 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:21.350 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:21.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:21.438 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:21.434+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:21.514 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:21.514 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:21.515 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:21.528 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:21.574 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:21.570+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.66726476900001, 2024-09-13T09:28:21.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:28:21.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.637 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 95.554045024999994, 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:28:21.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 95.553647010000006, 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:21.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878540134000005, 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878525202000006, 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:28:21.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878492096000002, 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:28:21.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878475803000001, 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:28:21.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878459688000007, 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:28:21.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878428647000007, 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:28:21.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:21.673 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.878397023999995, 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:28:21.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:28:21.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865448963000006, 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:28:21.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:28:21.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865431907000001, 2024-09-13T09:28:21.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:28:21.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865415738999999, 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.687 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:28:21.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865202077000006, 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865182766000004, 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:21.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865166372999994, 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:28:21.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865150710999998, 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.865124356999999, 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:28:21.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.33661163, 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:28:21.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:28:21.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:28:21.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.224697515000003, 2024-09-13T09:28:21.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:21.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:21.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:28:21.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:21.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.728 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:21.728 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:21.728 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:21.729 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:21.730 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:21.723+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:21.830 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:21.826+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:21.865 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:21.865 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:21.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.865 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:183526 3.2 3:4420c587:::rbd_object_map.11183cfd6e74:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-13T09:28:21.865 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:21.861991+0000", 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.002065772, 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00208625, 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 183526 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.861991+0000", 2024-09-13T09:28:21.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.861991+0000", 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.861988+0000", 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:21.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.862002+0000", 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3857e-05 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.862033+0000", 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0596999999999999e-05 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.862048+0000", 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5534000000000001e-05 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.862156+0000", 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000107305 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.862319+0000", 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016322299999999999 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:21.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 5", 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:21.862618+0000", 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00029873999999999998 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 1 2024-09-13T09:28:21.870 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:21.875 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:21.933 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:21.926+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:21.988 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:21.988 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:21.988 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:21.988 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:21.988 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:21.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:21.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:22.044 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:22.039+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:22.119 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 77.378373457999999, 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:22.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:22.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:22.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:22.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.126 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.136200707, 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:22.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:22.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.136177492000002, 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:22.131 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:22.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.132049174999999, 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:22.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.131996614999998, 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:22.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:22.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:22.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.13195992, 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.149 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.131940289999999, 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:22.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:22.154 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.131918111000001, 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:22.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:22.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.128162327999998, 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.159 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:22.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.128136993999998, 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:22.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.128119157, 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.168 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:22.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.128106771999999, 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:22.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.128078010999999, 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:22.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.123925799999999, 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:22.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.123897511999999, 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.186 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:22.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.190 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.123871378, 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:22.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:22.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.123846741000001, 2024-09-13T09:28:22.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:22.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.199 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.123764167000001, 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.200 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.123721264, 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:22.204 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:22.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 16.120095542000001, 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:22.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:22.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:22.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:22.213 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:22.215 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:22.215 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:22.215 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:22.215 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:22.216 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:22.216 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:22.207+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:22.228 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:22.229 INFO:teuthology.orchestra.run.smithi099.stderr:Exporting 3.13 info 3.13( v 131'967 (0'0,131'967] local-lis/les=107/108 n=82 ec=21/21 lis/c=139/107 les/c/f=140/108/0 sis=142 pruub=14.034248352s) 2024-09-13T09:28:22.251 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c802a827:::rbd_data.11186f48fbfa.000000000000001a:head# 2024-09-13T09:28:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c82b0d8e:::rbd_object_map.111811db2be7:head# 2024-09-13T09:28:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c83a3c61:::rbd_data.1118a28cd5c4.00000000000000d5:head# 2024-09-13T09:28:22.253 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c849ebfc:::rbd_data.111888f95246.0000000000000142:2e# 2024-09-13T09:28:22.254 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c849ebfc:::rbd_data.111888f95246.0000000000000142:head# 2024-09-13T09:28:22.254 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head# 2024-09-13T09:28:22.254 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c88fabb9:::rbd_data.1118413dc0c1.0000000000000002:22# 2024-09-13T09:28:22.269 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c88fabb9:::rbd_data.1118413dc0c1.0000000000000002:head# 2024-09-13T09:28:22.273 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:22.273 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:22.273 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:22.273 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:22.273 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:22.273 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:22.274 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:22.274 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:22.274 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:22.274 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c8a208f4:::rbd_data.11183cfd6e74.0000000000000005:2c# 2024-09-13T09:28:22.275 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c8a208f4:::rbd_data.11183cfd6e74.0000000000000005:head# 2024-09-13T09:28:22.275 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c8f87a6a:::rbd_object_map.1118e220a4bb.000000000000002f:head# 2024-09-13T09:28:22.275 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c901ac53:::rbd_data.11183e1674b5.000000000000008c:28# 2024-09-13T09:28:22.276 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c901ac53:::rbd_data.11183e1674b5.000000000000008c:head# 2024-09-13T09:28:22.277 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c92a794e:::rbd_header.11183cfd6e74:head# 2024-09-13T09:28:22.277 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:22.278 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9370868:::rbd_data.1118de6ae5af.0000000000000002:head# 2024-09-13T09:28:22.278 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9377fa0:::rbd_data.11186f48fbfa.0000000000000054:head# 2024-09-13T09:28:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c93e74f7:::rbd_data.1118a28cd5c4.0000000000000033:head# 2024-09-13T09:28:22.279 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9401fb4:::rbd_data.1118a28cd5c4.000000000000001b:head# 2024-09-13T09:28:22.280 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9a00536:::rbd_data.11182b78662d.000000000000001a:27# 2024-09-13T09:28:22.282 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9a00536:::rbd_data.11182b78662d.000000000000001a:head# 2024-09-13T09:28:22.282 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9a19000:::rbd_data.11187f961807.0000000000000131:head# 2024-09-13T09:28:22.283 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9ec5250:::rbd_data.1118e3e5e25d.0000000000000116:head# 2024-09-13T09:28:22.284 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:c9f831d8:::rbd_object_map.11182b78662d:head# 2024-09-13T09:28:22.284 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ca28514f:::rbd_data.11181500dc3e.000000000000000a:head# 2024-09-13T09:28:22.286 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ca5f58e2:::rbd_data.1118de6ae5af.0000000000000011:head# 2024-09-13T09:28:22.287 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ca6be701:::rbd_data.11181a204f0d.0000000000000000:head# 2024-09-13T09:28:22.293 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:a# 2024-09-13T09:28:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head# 2024-09-13T09:28:22.307 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:caa4782d:::rbd_data.11189fb5ddd6.0000000000000006:b# 2024-09-13T09:28:22.309 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:caa4782d:::rbd_data.11189fb5ddd6.0000000000000006:head# 2024-09-13T09:28:22.309 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cac088b3:::rbd_data.11186f48fbfa.0000000000000013:head# 2024-09-13T09:28:22.309 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:caca62dd:::rbd_header.11189fb5ddd6:head# 2024-09-13T09:28:22.310 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cadb44a6:::rbd_data.11181cfec05f.0000000000000057:2b# 2024-09-13T09:28:22.311 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cadb44a6:::rbd_data.11181cfec05f.0000000000000057:head# 2024-09-13T09:28:22.312 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cb0e70a3:::rbd_data.1118a28cd5c4.0000000000000055:head# 2024-09-13T09:28:22.313 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cb4b97a2:::rbd_data.11187f961807.0000000000000011:head# 2024-09-13T09:28:22.314 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cb83ed74:::rbd_data.1118e98b86fc.0000000000000025:d# 2024-09-13T09:28:22.315 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:22.310+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cb83ed74:::rbd_data.1118e98b86fc.0000000000000025:head# 2024-09-13T09:28:22.315 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cba06dad:::rbd_data.111812ca204d.000000000000005a:head# 2024-09-13T09:28:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cba38ae6:::rbd_data.111851704b2b.0000000000000000:9# 2024-09-13T09:28:22.316 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cba38ae6:::rbd_data.111851704b2b.0000000000000000:head# 2024-09-13T09:28:22.317 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cbd5a4ba:::rbd_data.1118292427f7.0000000000000003:2d# 2024-09-13T09:28:22.325 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cbd5a4ba:::rbd_data.1118292427f7.0000000000000003:head# 2024-09-13T09:28:22.325 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cbd9e129:::rbd_data.111817fd4ca8.0000000000000006:head# 2024-09-13T09:28:22.326 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cbf72b8d:::rbd_data.11181500dc3e.000000000000000c:head# 2024-09-13T09:28:22.327 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:22.327 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:22.328 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cc0ebb7c:::rbd_data.11183cfd6e74.000000000000001b:2c# 2024-09-13T09:28:22.328 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cc0ebb7c:::rbd_data.11183cfd6e74.000000000000001b:head# 2024-09-13T09:28:22.328 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cc14da0b:::rbd_data.11181cfec05f.000000000000003f:head# 2024-09-13T09:28:22.329 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cc52b400:::rbd_data.11181839c212.0000000000000027:head# 2024-09-13T09:28:22.329 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:d# 2024-09-13T09:28:22.332 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:head# 2024-09-13T09:28:22.332 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ccbe4cb7:::rbd_id.image_client.0-clone10:head# 2024-09-13T09:28:22.332 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ccbea3e2:::rbd_data.11181839c212.00000000000000fe:18# 2024-09-13T09:28:22.333 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ccbea3e2:::rbd_data.11181839c212.00000000000000fe:head# 2024-09-13T09:28:22.333 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ccf7f293:::rbd_data.1118e3e5e25d.000000000000000d:head# 2024-09-13T09:28:22.333 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd14de06:::rbd_data.1118e3e5e25d.0000000000000000:head# 2024-09-13T09:28:22.334 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:d# 2024-09-13T09:28:22.337 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:head# 2024-09-13T09:28:22.338 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd44ea1b:::rbd_data.11181cfec05f.000000000000005c:head# 2024-09-13T09:28:22.339 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd6172ed:::rbd_data.11181cfec05f.0000000000000108:head# 2024-09-13T09:28:22.340 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd6fcc14:::rbd_data.11181839c212.00000000000001a1:18# 2024-09-13T09:28:22.340 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd6fcc14:::rbd_data.11181839c212.00000000000001a1:head# 2024-09-13T09:28:22.340 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd78d78f:::rbd_data.1118de6ae5af.0000000000000018:head# 2024-09-13T09:28:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cd8043ef:::rbd_data.11181cfec05f.00000000000000ba:head# 2024-09-13T09:28:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cdbb9e26:::rbd_data.11181cfec05f.00000000000000b3:head# 2024-09-13T09:28:22.341 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cdca897a:::rbd_data.11185dd0f6b.0000000000000016:13# 2024-09-13T09:28:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cdca897a:::rbd_data.11185dd0f6b.0000000000000016:head# 2024-09-13T09:28:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cdd1c49b:::rbd_data.1118fc9bb66a.000000000000003f:head# 2024-09-13T09:28:22.345 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ce4788df:::rbd_data.111818d3d5a9.0000000000000002:15# 2024-09-13T09:28:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ce4788df:::rbd_data.111818d3d5a9.0000000000000002:head# 2024-09-13T09:28:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ce720d86:::rbd_data.11181839c212.000000000000034c:18# 2024-09-13T09:28:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ce720d86:::rbd_data.11181839c212.000000000000034c:head# 2024-09-13T09:28:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cebebb85:::rbd_data.11181839c212.00000000000000f8:18# 2024-09-13T09:28:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cebebb85:::rbd_data.11181839c212.00000000000000f8:head# 2024-09-13T09:28:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ceeea626:::rbd_data.1118172bd520.00000000000000f8:1c# 2024-09-13T09:28:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ceeea626:::rbd_data.1118172bd520.00000000000000f8:head# 2024-09-13T09:28:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cf5c2712:::rbd_object_map.111838d2a860.0000000000000017:head# 2024-09-13T09:28:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cf78d74f:::rbd_data.11182b78662d.0000000000000004:27# 2024-09-13T09:28:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cf78d74f:::rbd_data.11182b78662d.0000000000000004:head# 2024-09-13T09:28:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cf91c03c:::rbd_data.11183cfd6e74.000000000000001d:2c# 2024-09-13T09:28:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cf91c03c:::rbd_data.11183cfd6e74.000000000000001d:head# 2024-09-13T09:28:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:cff1fea6:::rbd_data.1118fc9bb66a.0000000000000015:head# 2024-09-13T09:28:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:Export successful 2024-09-13T09:28:22.417 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:22.414+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:22.446 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:22.446 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:22.456 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:22.518 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:22.511+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:22.567 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:22.567 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:22.577 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:22.619 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:22.611+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:22.686 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:22.687 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:22.697 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:22.741 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:22.735+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:22.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:28:22.819 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:22.811+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:22.871 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:22.871 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:22.883 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:22.924 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:22.919+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:22.934 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.13 2024-09-13T09:28:23.022 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:23.018+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:23.042 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.042 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.053 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:23.123 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:23.122+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:23.152 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.152 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.161 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:23.224 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:23.218+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:23.268 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.268 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.279 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:23.325 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:23.318+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:23.389 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.389 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:23.426 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:23.419+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:23.507 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.507 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:23.527 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:23.522+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:23.625 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.625 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.628 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:23.623+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:23.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:23.729 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:23.722+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:23.737 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.738 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.747 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:23.787 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:28:23.829 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:23.826+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:23.908 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:23.908 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:23.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:23.961 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:23.958+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:24.019 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:24.019 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:24.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:24.030 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:24.023+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:24.132 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:24.126+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:24.133 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:24.133 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: marking collection for removal 2024-09-13T09:28:24.138 INFO:teuthology.orchestra.run.smithi099.stdout:setting '_remove' omap key 2024-09-13T09:28:24.139 INFO:teuthology.orchestra.run.smithi099.stdout:finish_remove_pgs 3.13_head removing 3.13 2024-09-13T09:28:24.143 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:24.163 INFO:teuthology.orchestra.run.smithi099.stdout:Remove successful 2024-09-13T09:28:24.233 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:24.230+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:24.264 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:24.264 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:24.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:24.279 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:24.334 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:24.330+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:24.377 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:24.377 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:24.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:24.434 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:24.434 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:24.427+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:24.455 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:24.480 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:24.493 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:24.493 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:24.500 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:24.503 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:24.523 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:24.535 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:24.530+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:24.545 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:24.568 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:24.568 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:24.568 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:24.569 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:24.569 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:24.569 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:24.569 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:24.569 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:24.569 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:24.572 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:24.573 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:24.614 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:24.615 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:24.624 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:24.668 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:24.663+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:24.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.13.4 2024-09-13T09:28:24.737 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:24.731+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:24.779 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:24.780 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:24.780 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:24.780 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:24.780 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:24.789 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:24.838 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:24.831+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:24.906 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:24.906 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 80.162858021999995, 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:24.907 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.908 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:24.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:24.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.911 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.911 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:24.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:24.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:24.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:24.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:24.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:24.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:24.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:24.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:24.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:24.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:24.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:24.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:24.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.920685271, 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.921 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.922 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:24.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.922 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.920662056000001, 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.927 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.931 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.916533738999998, 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.932 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:24.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.934 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.934 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:24.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:24.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:24.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:24.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:24.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.938 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.938 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:24.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:24.939 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.939 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.939 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.940 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.940 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:24.940 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.916481179000002, 2024-09-13T09:28:24.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.941 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:24.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.944 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:24.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.944 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.945 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:24.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:24.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.916444483999999, 2024-09-13T09:28:24.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.952 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.953 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.953 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.953 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:24.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:24.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:24.957 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.916424853999999, 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.958 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.959 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:24.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.959 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:24.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:24.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:24.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:24.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.916402675, 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:24.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:24.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.912646892000001, 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.971 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:24.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.912621558000001, 2024-09-13T09:28:24.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:24.980 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.912603721, 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:24.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:24.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:24.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.912591335999998, 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:24.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:24.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:24.990 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.912562574999999, 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:24.994 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.908410364000002, 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:24.995 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:24.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:24.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:24.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:24.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:24.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:25.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:25.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.908382075999999, 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:25.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:25.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.908355942, 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:25.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:25.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:25.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.908331305000001, 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:25.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.908248731, 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:25.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:25.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.908205828, 2024-09-13T09:28:25.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:25.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.025 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.904580106000001, 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:25.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:25.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.030 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.031 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.031 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:25.032 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:25.027+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:25.033 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:25.133 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:25.130+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:25.197 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:25.197 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:25.197 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:25.198 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:25.209 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:25.233 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:25.227+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:25.320 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:25.320 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:25.320 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:25.331 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:25.334 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:25.330+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:25.435 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:25.427+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:25.490 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:25.490 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:25.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:25.490 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 103.924271804, 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:25.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:25.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:25.497 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 71.336613424999996, 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:25.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.206321299999999, 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:25.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:25.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:25.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:25.507 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.622110848999998, 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.508 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:25.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220777216999998, 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:25.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:25.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220759209000001, 2024-09-13T09:28:25.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:25.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220741870000001, 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.522 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:25.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:25.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220724613000002, 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.527 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220707077, 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220690007000002, 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.536 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:25.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:25.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:25.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:25.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:25.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.541 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220676348000001, 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.542 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:25.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:25.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.547 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220666675, 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.548 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:25.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:25.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.552 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220652130000001, 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.553 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:25.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:25.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.557 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220635463000001, 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.558 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:25.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.220618372000001, 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:25.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.567 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.218762796, 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:25.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:25.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.572 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.218746688, 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.573 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:25.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:25.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.218732324000001, 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:25.578 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:25.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:25.583 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.218718496000001, 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:25.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:25.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 18.218700796, 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:25.588 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:25.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:25.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:25.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.593 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.593 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:25.593 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:25.593 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:25.595 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:25.595 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:25.587+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:25.695 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:25.687+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:25.757 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:25.757 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:25.757 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:25.757 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:25.767 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:25.801 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:28:25.812 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:25.807+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:25.878 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:25.878 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:25.878 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:25.878 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:25.879 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:25.888 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:25.897 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:25.890+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:25.901 INFO:teuthology.orchestra.run.smithi099.stdout:get_pg_num_history pg_num_history pg_num_history(e142 pg_nums {1={16=8},2={17=1},3={21=32}} deleted_pools ) 2024-09-13T09:28:25.901 INFO:teuthology.orchestra.run.smithi099.stdout:Importing pgid 3.13 2024-09-13T09:28:25.908 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c802a827:::rbd_data.11186f48fbfa.000000000000001a:head# 2024-09-13T09:28:25.908 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.909 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c82b0d8e:::rbd_object_map.111811db2be7:head# 2024-09-13T09:28:25.909 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.910 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c83a3c61:::rbd_data.1118a28cd5c4.00000000000000d5:head# 2024-09-13T09:28:25.911 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.912 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c849ebfc:::rbd_data.111888f95246.0000000000000142:2e# 2024-09-13T09:28:25.914 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c849ebfc:::rbd_data.111888f95246.0000000000000142:head# 2024-09-13T09:28:25.914 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2e=[]:{2e=[2e]} 2024-09-13T09:28:25.915 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head# 2024-09-13T09:28:25.915 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.916 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c88fabb9:::rbd_data.1118413dc0c1.0000000000000002:22# 2024-09-13T09:28:25.937 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c88fabb9:::rbd_data.1118413dc0c1.0000000000000002:head# 2024-09-13T09:28:25.938 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 22=[]:{22=[22]} 2024-09-13T09:28:25.943 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c8a208f4:::rbd_data.11183cfd6e74.0000000000000005:2c# 2024-09-13T09:28:25.944 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c8a208f4:::rbd_data.11183cfd6e74.0000000000000005:head# 2024-09-13T09:28:25.944 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2c=[]:{2c=[2c]} 2024-09-13T09:28:25.945 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c8f87a6a:::rbd_object_map.1118e220a4bb.000000000000002f:head# 2024-09-13T09:28:25.945 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.945 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c901ac53:::rbd_data.11183e1674b5.000000000000008c:28# 2024-09-13T09:28:25.946 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c901ac53:::rbd_data.11183e1674b5.000000000000008c:head# 2024-09-13T09:28:25.946 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 28=[]:{28=[28]} 2024-09-13T09:28:25.947 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c92a794e:::rbd_header.11183cfd6e74:head# 2024-09-13T09:28:25.947 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.947 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9370868:::rbd_data.1118de6ae5af.0000000000000002:head# 2024-09-13T09:28:25.947 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.948 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9377fa0:::rbd_data.11186f48fbfa.0000000000000054:head# 2024-09-13T09:28:25.948 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 16=[]:{} 2024-09-13T09:28:25.949 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c93e74f7:::rbd_data.1118a28cd5c4.0000000000000033:head# 2024-09-13T09:28:25.949 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.949 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9401fb4:::rbd_data.1118a28cd5c4.000000000000001b:head# 2024-09-13T09:28:25.949 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.950 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9a00536:::rbd_data.11182b78662d.000000000000001a:27# 2024-09-13T09:28:25.951 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9a00536:::rbd_data.11182b78662d.000000000000001a:head# 2024-09-13T09:28:25.951 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 27=[]:{27=[27]} 2024-09-13T09:28:25.952 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9a19000:::rbd_data.11187f961807.0000000000000131:head# 2024-09-13T09:28:25.952 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.953 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9ec5250:::rbd_data.1118e3e5e25d.0000000000000116:head# 2024-09-13T09:28:25.953 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.953 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:c9f831d8:::rbd_object_map.11182b78662d:head# 2024-09-13T09:28:25.953 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.954 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ca28514f:::rbd_data.11181500dc3e.000000000000000a:head# 2024-09-13T09:28:25.954 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.955 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ca5f58e2:::rbd_data.1118de6ae5af.0000000000000011:head# 2024-09-13T09:28:25.955 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.955 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ca6be701:::rbd_data.11181a204f0d.0000000000000000:head# 2024-09-13T09:28:25.956 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.959 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:a# 2024-09-13T09:28:25.969 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head# 2024-09-13T09:28:25.972 INFO:teuthology.orchestra.run.smithi099.stdout:snapset a=[]:{a=[a]} 2024-09-13T09:28:25.978 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:caa4782d:::rbd_data.11189fb5ddd6.0000000000000006:b# 2024-09-13T09:28:25.981 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:caa4782d:::rbd_data.11189fb5ddd6.0000000000000006:head# 2024-09-13T09:28:25.981 INFO:teuthology.orchestra.run.smithi099.stdout:snapset b=[]:{b=[b]} 2024-09-13T09:28:25.981 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cac088b3:::rbd_data.11186f48fbfa.0000000000000013:head# 2024-09-13T09:28:25.981 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.982 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:caca62dd:::rbd_header.11189fb5ddd6:head# 2024-09-13T09:28:25.982 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.982 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cadb44a6:::rbd_data.11181cfec05f.0000000000000057:2b# 2024-09-13T09:28:25.983 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cadb44a6:::rbd_data.11181cfec05f.0000000000000057:head# 2024-09-13T09:28:25.983 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2b=[]:{2b=[2b]} 2024-09-13T09:28:25.984 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cb0e70a3:::rbd_data.1118a28cd5c4.0000000000000055:head# 2024-09-13T09:28:25.984 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.985 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cb4b97a2:::rbd_data.11187f961807.0000000000000011:head# 2024-09-13T09:28:25.985 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.986 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cb83ed74:::rbd_data.1118e98b86fc.0000000000000025:d# 2024-09-13T09:28:25.987 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cb83ed74:::rbd_data.1118e98b86fc.0000000000000025:head# 2024-09-13T09:28:25.987 INFO:teuthology.orchestra.run.smithi099.stdout:snapset d=[]:{d=[d]} 2024-09-13T09:28:25.988 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cba06dad:::rbd_data.111812ca204d.000000000000005a:head# 2024-09-13T09:28:25.988 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:25.988 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cba38ae6:::rbd_data.111851704b2b.0000000000000000:9# 2024-09-13T09:28:25.989 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cba38ae6:::rbd_data.111851704b2b.0000000000000000:head# 2024-09-13T09:28:25.990 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 9=[]:{9=[9]} 2024-09-13T09:28:25.991 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cbd5a4ba:::rbd_data.1118292427f7.0000000000000003:2d# 2024-09-13T09:28:25.997 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417325611999999, 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417313374999999, 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417297978000001, 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417283103000003, 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417265894000003, 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417245438999998, 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417227359000002, 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417210982999997, 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417195632000002, 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417178927999998, 2024-09-13T09:28:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417160846000002, 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417144100999998, 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.417099442000001, 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.414150002, 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.414129463999998, 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.414106848999999, 2024-09-13T09:28:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.414087981000002, 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.414053387000003, 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:26.079 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.409774482000003, 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:26.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:26.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 38.409710509999996, 2024-09-13T09:28:26.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:26.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.092 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.093 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.093 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.093 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.093 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:26.093 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:25.991+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:26.094 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:26.101 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cbd5a4ba:::rbd_data.1118292427f7.0000000000000003:head# 2024-09-13T09:28:26.101 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2d=[]:{2d=[2d]} 2024-09-13T09:28:26.101 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cbd9e129:::rbd_data.111817fd4ca8.0000000000000006:head# 2024-09-13T09:28:26.101 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cbf72b8d:::rbd_data.11181500dc3e.000000000000000c:head# 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cc0ebb7c:::rbd_data.11183cfd6e74.000000000000001b:2c# 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cc0ebb7c:::rbd_data.11183cfd6e74.000000000000001b:head# 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2c=[]:{2c=[2c]} 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cc14da0b:::rbd_data.11181cfec05f.000000000000003f:head# 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cc52b400:::rbd_data.11181839c212.0000000000000027:head# 2024-09-13T09:28:26.102 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:d# 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cc7b1844:::rbd_data.1118e98b86fc.0000000000000006:head# 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:snapset d=[]:{d=[d]} 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ccbe4cb7:::rbd_id.image_client.0-clone10:head# 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ccbea3e2:::rbd_data.11181839c212.00000000000000fe:18# 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ccbea3e2:::rbd_data.11181839c212.00000000000000fe:head# 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 18=[]:{18=[18]} 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ccf7f293:::rbd_data.1118e3e5e25d.000000000000000d:head# 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.103 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd14de06:::rbd_data.1118e3e5e25d.0000000000000000:head# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:d# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd2d0eeb:::rbd_data.1118e98b86fc.0000000000000002:head# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:snapset d=[]:{d=[d]} 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd44ea1b:::rbd_data.11181cfec05f.000000000000005c:head# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd6172ed:::rbd_data.11181cfec05f.0000000000000108:head# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd6fcc14:::rbd_data.11181839c212.00000000000001a1:18# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd6fcc14:::rbd_data.11181839c212.00000000000001a1:head# 2024-09-13T09:28:26.104 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 18=[]:{18=[18]} 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd78d78f:::rbd_data.1118de6ae5af.0000000000000018:head# 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cd8043ef:::rbd_data.11181cfec05f.00000000000000ba:head# 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cdbb9e26:::rbd_data.11181cfec05f.00000000000000b3:head# 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cdca897a:::rbd_data.11185dd0f6b.0000000000000016:13# 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cdca897a:::rbd_data.11185dd0f6b.0000000000000016:head# 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 13=[]:{13=[13]} 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cdd1c49b:::rbd_data.1118fc9bb66a.000000000000003f:head# 2024-09-13T09:28:26.105 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ce4788df:::rbd_data.111818d3d5a9.0000000000000002:15# 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ce4788df:::rbd_data.111818d3d5a9.0000000000000002:head# 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 15=[]:{15=[15]} 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ce720d86:::rbd_data.11181839c212.000000000000034c:18# 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ce720d86:::rbd_data.11181839c212.000000000000034c:head# 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 18=[]:{18=[18]} 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cebebb85:::rbd_data.11181839c212.00000000000000f8:18# 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cebebb85:::rbd_data.11181839c212.00000000000000f8:head# 2024-09-13T09:28:26.106 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 18=[]:{18=[18]} 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ceeea626:::rbd_data.1118172bd520.00000000000000f8:1c# 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:ceeea626:::rbd_data.1118172bd520.00000000000000f8:head# 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 1c=[]:{1c=[1c]} 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cf5c2712:::rbd_object_map.111838d2a860.0000000000000017:head# 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cf78d74f:::rbd_data.11182b78662d.0000000000000004:27# 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cf78d74f:::rbd_data.11182b78662d.0000000000000004:head# 2024-09-13T09:28:26.107 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 27=[]:{27=[27]} 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cf91c03c:::rbd_data.11183cfd6e74.000000000000001d:2c# 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cf91c03c:::rbd_data.11183cfd6e74.000000000000001d:head# 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2c=[]:{2c=[2c]} 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:cff1fea6:::rbd_data.1118fc9bb66a.0000000000000015:head# 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:write_pg epoch 142 info 3.13( v 131'967 (0'0,131'967] local-lis/les=107/108 n=82 ec=21/21 lis/c=139/107 les/c/f=140/108/0 sis=142 pruub=14.034248352s) 2024-09-13T09:28:26.108 INFO:teuthology.orchestra.run.smithi099.stdout:Import successful 2024-09-13T09:28:26.108 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:26.095+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:26.183 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:28:26.201 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:26.195+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:26.220 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:26.220 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:26.221 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:26.221 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:26.231 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:26.303 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:26.298+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:26.341 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:26.342 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:26.342 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:26.342 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:26.342 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:26.351 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:26.403 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:26.398+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:26.452 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.890002137, 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:26.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:26.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:26.457 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.261634563000001, 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:26.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:26.462 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.220055969000001, 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:26.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.467 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218640789999998, 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:26.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218622137000001, 2024-09-13T09:28:26.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:26.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218549588999998, 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.477 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:26.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218312814000001, 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.482 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:26.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.486 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218269982999999, 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.487 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:26.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218246320999999, 2024-09-13T09:28:26.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:26.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:26.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218036290000001, 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.496 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:26.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:26.500 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.218009419000001, 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.501 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.505 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217991524999999, 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.506 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:26.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:26.510 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217975278000001, 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:26.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217877214000001, 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.515 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:26.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.519 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217643979000002, 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217549372000001, 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:26.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217523846999999, 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:26.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.534 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217500668, 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:26.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:26.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:26.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217226645, 2024-09-13T09:28:26.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:26.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 19.217042494000001, 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:26.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:26.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:26.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:26.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:26.549 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:26.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:26.551 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:26.546+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:26.573 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:26.611 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /var/log/ceph/exp.3.13.4 2024-09-13T09:28:26.651 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:26.646+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:26.669 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:26.669 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:26.670 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:26.670 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:26.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:26.728 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:26.748 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:26.752 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:26.747+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:26.775 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:26.783 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:26.783 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:26.783 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:26.784 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:26.784 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:26.793 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:26.796 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:26.818 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:26.840 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:26.852 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:26.847+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:26.863 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:26.863 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:26.864 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:26.864 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:26.864 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:26.864 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:26.864 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:26.864 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:26.867 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:26.868 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:26.901 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:26.901 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:26.911 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:26.953 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:26.947+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:27.019 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.019 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:27.054 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:27.051+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:27.138 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.138 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:27.154 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:27.151+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:27.255 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:27.247+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:27.261 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.262 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.272 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:27.356 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:27.351+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:27.377 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.377 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:27.458 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:27.451+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:27.493 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.493 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:27.559 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:27.554+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:27.608 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.608 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.618 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:27.665 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:27.659+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:27.738 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.738 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.748 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:27.760 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:27.758+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:27.853 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.853 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.861 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:27.855+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:27.864 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:27.962 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:27.955+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:27.968 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:27.968 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:27.978 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:28.064 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:28.059+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:28.088 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.088 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.098 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:28.165 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:28.162+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:28.201 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:28:28.214 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.214 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.228 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:28.273 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:28.267+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:28.367 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:28.359+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:28.389 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.389 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.399 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:28.468 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:28.462+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:28.503 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.503 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.513 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:28.569 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:28.563+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:28.621 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.622 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:28.670 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:28.666+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:28.742 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.742 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:28.796 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:28.791+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:28.861 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.861 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.869 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:28.872 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:28.873 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:28.866+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:28.973 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:28.970+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:28.988 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:28.998 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:29.017 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:29.041 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:29.058 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:29.074 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:29.067+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:29.078 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:29.102 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:29.105 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.543074469, 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:29.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:29.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.914706894999998, 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.112 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:29.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.873128301000001, 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:29.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:29.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:29.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.121 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871713121999999, 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.122 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:29.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871694469000001, 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.127 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:29.131 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871621920999999, 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.132 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:29.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:29.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871385146000001, 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:29.137 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:29.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:29.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.142 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871342315, 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:29.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:29.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:29.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:29.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.148 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871318652999999, 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871108622000001, 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.153 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:29.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:29.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:29.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871081750999998, 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:29.159 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:29.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871063856999999, 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.170 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.871047610000002, 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:29.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870949545999999, 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.175 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870716310999999, 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870621704000001, 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870596179, 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870573, 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870298977000001, 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 21.870114825999998, 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:29.210 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:29.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:29.213 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:29.213 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:29.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:29.214 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:28:29.215 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:29.206+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:29.313 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:29.306+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:29.328 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:29.328 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:29.328 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:29.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:29.414 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:29.407+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:29.445 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:29.445 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:29.445 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:29.445 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:29.445 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:29.455 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:29.515 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:29.507+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:29.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 97.475198950000006, 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:29.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.251958359, 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:29.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:29.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:29.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:29.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 74.724745153000001, 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.573 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:29.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:29.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.465498203999999, 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:29.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:29.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.465481649000001, 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.465466585000001, 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:29.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:29.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.464734822, 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.594 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:29.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:29.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.464710916000001, 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.598 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:29.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.464018916000001, 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.464004880000001, 2024-09-13T09:28:29.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:29.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.463886503000001, 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.612 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:29.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.463693371000002, 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:29.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:29.621 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459758251, 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:29.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:29.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459749564999999, 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:29.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:29.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459740204999999, 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:29.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:29.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.635 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459731426000001, 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:29.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:29.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459722846999998, 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:29.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:29.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459710163, 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.645 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:29.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.649 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.459701271, 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:29.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:29.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:182783 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-13T09:28:29.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.893874395000001, 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99356371799999998, 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 182783 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:29.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663410+0000", 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663420+0000", 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0097e-05 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663440+0000", 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9559999999999999e-05 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663448+0000", 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.824e-06 2024-09-13T09:28:29.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663515+0000", 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6964999999999993e-05 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663527+0000", 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2295e-05 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656700+0000", 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99317316300000003 2024-09-13T09:28:29.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656806+0000", 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105889 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656976+0000", 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016978699999999999 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:29.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:29.660 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:29.661 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:29.661 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:29.658+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:29.762 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:29.758+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:29.772 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:29.772 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:29.772 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:29.772 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:29.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:29.863 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:29.859+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:29.890 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:29.890 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:29.891 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:29.891 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:29.891 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:29.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:29.964 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:29.958+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:30.008 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:30.008 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:30.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:30.008 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.580241143999999, 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.009 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:30.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.580226088, 2024-09-13T09:28:30.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.014 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:30.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.580216679999999, 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:30.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:30.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:30.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.580202387, 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:30.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:30.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.580116681, 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:30.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579760432, 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:30.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:30.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579713596000001, 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.038 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:30.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579674610000001, 2024-09-13T09:28:30.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:30.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:30.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:30.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579654466000001, 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:30.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:30.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579635398000001, 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:30.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:30.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:30.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:30.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:30.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579593309, 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:30.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:30.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:30.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579479806999998, 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:30.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:30.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579461628000001, 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:30.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579451883000001, 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:30.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579441696, 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:30.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:30.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579348830000001, 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:30.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.579245417999999, 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:30.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:30.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.578923540000002, 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:30.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.093 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.578507599000002, 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:30.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:30.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.578253888999999, 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:30.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:30.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.103 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:30.104 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:30.105 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:30.102+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:30.205 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:30.199+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:30.264 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:30.264 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:30.264 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:30.264 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:30.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:30.306 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:30.302+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:30.378 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:30.378 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:30.379 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:30.379 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:30.379 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:30.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:30.407 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:30.399+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:30.498 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 167.53412627899999, 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:30.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:28:30.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.420906535, 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:28:30.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.42050852, 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.509 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:28:30.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.745401644, 2024-09-13T09:28:30.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:28:30.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:28:30.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:28:30.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:28:30.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.745386712, 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:28:30.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.74535360599999, 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:30.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.74533731299999, 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:28:30.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:28:30.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.745321198, 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:28:30.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.745290157, 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:28:30.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.745258534, 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:28:30.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.732310473, 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.73229341699999, 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:28:30.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.73227724900001, 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:28:30.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:28:30.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.732063587, 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:28:30.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.732044276, 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:28:30.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:28:30.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:28:30.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.732027883, 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.73201222100001, 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:28:30.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:28:30.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.73198586700001, 2024-09-13T09:28:30.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:28:30.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.20347314, 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:28:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:28:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:28:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:28:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 52.091559025000002, 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:28:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:30.601 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:30.603 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:30.603 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:30.598+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:30.704 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:30.695+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:30.724 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:30.725 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:30.725 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:30.725 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:30.732 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:30.804 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:30.799+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:30.838 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:30.838 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:30.839 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:30.839 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:30.839 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:30.848 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:30.905 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:30.899+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 86.217677619, 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:30.960 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.961 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:30.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:30.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:30.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.975504868000002, 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:30.967 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:30.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:30.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.971 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.975481652999999, 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.971353336, 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.971300776, 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:30.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:30.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.985 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.971264081000001, 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:30.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.990 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.971244451, 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:30.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.971222271999999, 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:30.995 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:30.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:30.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:30.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.967466489, 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:31.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.967441154999999, 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.005 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:31.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:31.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.967423318000002, 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.009 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.013 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.967410933, 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:31.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.967382172000001, 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.018 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:31.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.022 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.963229961, 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.023 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.963201673, 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.028 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:31.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:31.032 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.963175539000002, 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:31.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.963150901999999, 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:31.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:31.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.963068327999999, 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:31.042 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:31.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:31.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:31.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.963025425000001, 2024-09-13T09:28:31.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:31.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.051 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 24.959399702999999, 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:31.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:31.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:31.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:31.056 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:31.058 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:31.058 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:31.054+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:31.159 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:31.158+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:31.175 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:31.176 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:31.185 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:31.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:31.220 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:28:31.260 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:31.255+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:31.325 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:31.326 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:31.326 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:31.326 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:31.326 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:31.340 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:31.360 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:31.354+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:31.365 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:31.394 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:31.422 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:31.446 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:31.456 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:31.456 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:31.461 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:31.458+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:31.466 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:31.469 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:31.497 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:31.519 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:31.520 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:31.520 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:31.523 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:31.523 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:31.562 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:31.555+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:31.585 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:31.585 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:31.595 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:31.644 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0] out_osds: [4, 2] dead_osds: [4] live_osds: [0, 1, 2, 5, 6, 7, 3] 2024-09-13T09:28:31.644 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:28:31.644 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-13T09:28:31.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-13T09:28:31.663 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:31.655+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:31.748 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:31.748 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:31.758 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:31.764 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:31.758+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:31.865 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:31.859+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:31.905 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:31.905 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:31.915 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:31.955 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:28:31.947+0000 7f00df3a9700 -1 mon.a@0(leader).osd e156 definitely_dead 0 2024-09-13T09:28:31.966 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:31.962+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:32.021 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.022 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.031 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:32.072 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:32.067+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:32.168 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:32.163+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:32.185 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.185 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.195 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:32.235 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:28:32.242 INFO:teuthology.orchestra.run.smithi063.stderr:marked in osd.2. 2024-09-13T09:28:32.256 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-13T09:28:32.268 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:32.262+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:32.313 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.314 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:32.371 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:32.366+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:32.438 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.439 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.449 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:32.471 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:32.466+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:32.556 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.557 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:32.610 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:32.606+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:32.674 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:32.670+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:32.718 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.718 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:32.774 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:32.771+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:32.834 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.834 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:32.874 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:32.871+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:32.953 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:32.953 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:32.963 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:32.975 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:32.969+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:33.073 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:33.073 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:33.076 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:33.071+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:33.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:33.178 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:33.175+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:33.193 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:33.194 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:33.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:33.278 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:33.271+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:33.312 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:33.312 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:33.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:33.379 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:33.375+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:33.425 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:33.425 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:33.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:33.481 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:33.475+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:33.524 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:33.556 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:33.556 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:33.566 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:33.582 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:33.577+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:33.669 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:33.672 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:33.672 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:33.682 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:33.683 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:33.675+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:33.687 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:33.709 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:33.729 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:33.752 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:33.781 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:33.783 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:33.775+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.053011858000005, 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:33.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.799 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.810839107, 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.810815892000001, 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.804 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:33.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.806687575000002, 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:33.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:33.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.806635015000001, 2024-09-13T09:28:33.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:33.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:33.818 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.806598319999999, 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:33.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:33.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:33.823 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.806578689999998, 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:33.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:33.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.806556511, 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:33.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.802800728000001, 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.802775394000001, 2024-09-13T09:28:33.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:33.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.802757557, 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:33.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:33.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.802745172000002, 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.847 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:33.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:33.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.802716410999999, 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.798564200000001, 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.857 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:33.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:33.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.798535911999998, 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:33.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:33.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:33.865 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.798509778, 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:33.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.870 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.798485141, 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:33.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.875 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.798402567, 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:33.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.798359663999999, 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:33.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:33.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.794733942000001, 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:33.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:33.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:33.889 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:33.890 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:33.891 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:33.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:33.892 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:33.892 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:33.893 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:33.894 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:33.885+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:33.992 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:33.985+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:34.006 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:34.006 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:34.006 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:34.016 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:34.093 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:34.089+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:34.127 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:34.128 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:34.128 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:34.128 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:34.128 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:34.137 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:34.194 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:34.189+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:34.249 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.692877126, 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:34.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:34.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:34.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 80.105218746999995, 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:34.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.267 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.268 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.974926622000002, 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:34.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 27.390716171000001, 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.273 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:34.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.277 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989382539000001, 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:34.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:34.281 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989364531, 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:34.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:34.286 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989347192, 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.290 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989329935000001, 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.291 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:34.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989312398999999, 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:34.295 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:34.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:34.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989295329000001, 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.300 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:34.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:34.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.98928167, 2024-09-13T09:28:34.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:34.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:34.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.309 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989271996999999, 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:34.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:34.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989257452, 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:34.314 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:34.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989240785, 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.989223694, 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.987368117999999, 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.987352009999999, 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.987337646, 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.987323818, 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 26.987306117999999, 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.351 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:34.353 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:34.353 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:34.349+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:34.453 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:34.447+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:34.496 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:34.496 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:34.497 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:34.497 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:34.508 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:34.554 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:34.547+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:34.656 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:34.647+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:34.662 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:34.662 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:34.662 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:34.662 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:34.662 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:34.672 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:34.757 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:34.751+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:34.792 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212252175000003, 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212239938000003, 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212224540999998, 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212209666, 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212192457, 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212172002000003, 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212153921999999, 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212137546000001, 2024-09-13T09:28:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212122194999999, 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212105491000003, 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212087408999999, 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212070664000002, 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.212026004999998, 2024-09-13T09:28:34.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:34.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.209076564999997, 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:34.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:34.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:34.860 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.209056027000003, 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.861 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:34.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.209033411999997, 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:34.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:34.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.209014543999999, 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:34.870 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:34.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:34.874 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.20897995, 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.875 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.879 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.204701045, 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:34.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:34.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:34.884 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.204637073000001, 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:34.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:34.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:34.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:34.889 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:34.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:34.891 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:34.889+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:34.992 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:34.985+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:35.063 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:35.063 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:35.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.063 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "MOSDPGPush(3.18 159/158 [PushOp(3:1b575f7c:::rbd_data.111812ca204d.0000000000000011:head, version: 46'134, data_included: [946176~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:1b575f7c:::rbd_data.111812ca204d.0000000000000011:head@46'134, size: 950272, copy_subset: [0~950272], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:950272, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:35.061462+0000", 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.00017830400000000001, 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00029221, 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061462+0000", 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061462+0000", 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061461+0000", 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061513+0000", 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2135000000000001e-05 2024-09-13T09:28:35.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061541+0000", 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8615e-05 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061561+0000", 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9293999999999999e-05 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061624+0000", 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3063000000000003e-05 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:35.061631+0000", 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5170000000000002e-06 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 1 2024-09-13T09:28:35.067 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:35.078 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:35.093 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:35.087+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:35.184 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:28:35.193 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:35.187+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:35.204 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:35.204 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:35.204 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:35.204 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:35.204 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:35.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:35.294 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:35.287+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:35.385 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:35.385 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:35.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:35.385 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:35.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.385 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.823408889, 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:35.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:35.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:35.390 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.195041315000001, 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:35.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:35.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.153462721, 2024-09-13T09:28:35.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:35.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.399 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.152047541999998, 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.400 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:35.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.404 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.152028889, 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:35.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:35.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.409 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151956341000002, 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.413 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151719566000001, 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:35.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:35.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:35.418 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151676734999999, 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151653072999999, 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.423 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:35.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151443042, 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:35.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:35.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151416171000001, 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:35.432 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:35.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:35.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:35.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:35.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.440 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.440 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.440 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.440 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151398276999998, 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:35.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:35.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151382030000001, 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.445 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:35.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151283966000001, 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.450 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.454 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.151050731000002, 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.455 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.459 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.150956124, 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:35.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.150930598999999, 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.150907419999999, 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:35.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.150633397, 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:35.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.477 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 28.150449246000001, 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:35.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:35.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:35.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:35.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:35.482 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:35.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:35.484 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:35.477+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:35.584 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:35.577+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:35.604 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:35.604 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:35.605 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:35.605 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:35.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:35.686 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:35.679+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:35.732 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:35.732 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:35.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:35.787 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:35.779+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:35.851 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:35.851 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:35.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:35.851 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:35.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.764841343, 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:35.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:35.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:35.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.541600751999994, 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:35.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:35.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:35.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.014387545999995, 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.863 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:35.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:35.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.755140597, 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:35.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.755124041999998, 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:35.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:35.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:35.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:35.878 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.755108977999999, 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:35.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:35.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.754377215000002, 2024-09-13T09:28:35.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:35.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.754353308999999, 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.888 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:35.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.753661309000002, 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:35.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:35.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:35.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.753647272999999, 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:35.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:35.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.753528895999999, 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.753335763999999, 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:35.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:35.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749400644000001, 2024-09-13T09:28:35.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:35.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:35.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749391958, 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.918 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:35.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749382598, 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:35.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.927 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749373818999999, 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:35.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749365239999999, 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:35.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:35.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749352556000002, 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.937 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:35.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 29.749343664000001, 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:35.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:35.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:35.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:182783 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:35.946 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.183516787999999, 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99356371799999998, 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 182783 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:35.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663410+0000", 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663420+0000", 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0097e-05 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663440+0000", 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9559999999999999e-05 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663448+0000", 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.824e-06 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663515+0000", 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6964999999999993e-05 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663527+0000", 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2295e-05 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656700+0000", 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99317316300000003 2024-09-13T09:28:35.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656806+0000", 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105889 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656976+0000", 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016978699999999999 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:35.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:35.952 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:35.952 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:35.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:35.954 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:35.945+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:36.054 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:36.047+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:36.073 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:36.073 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:36.073 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:36.073 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:36.084 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:36.104 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:36.134 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:36.154 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:36.147+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:36.161 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:36.192 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:36.196 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:36.196 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:36.196 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:36.196 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:36.196 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:36.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:36.223 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:36.249 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:36.255 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:36.247+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:36.271 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:36.271 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:36.272 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:36.276 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:36.277 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:36.318 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:36.318 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:36.328 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:36.357 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:36.351+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:36.434 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:36.434 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:36.444 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:36.457 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:36.453+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:36.546 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:36.546 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:36.556 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:36.558 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:36.553+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:36.658 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:36.651+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:36.661 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:36.661 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:36.671 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:36.759 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:36.751+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:36.805 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:36.805 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:36.815 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:36.860 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:36.853+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:36.934 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:36.935 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:36.948 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:36.992 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:36.987+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:37.062 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:37.055+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:37.069 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.069 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.080 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:37.163 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:37.155+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:37.195 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:28:37.205 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.205 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.217 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:37.257 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0, 2] out_osds: [4] dead_osds: [4] live_osds: [0, 1, 2, 5, 6, 7, 3] 2024-09-13T09:28:37.257 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:28:37.257 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T09:28:37.258 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:28:37.258 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:28:37.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T09:28:37.264 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:37.257+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:37.342 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.342 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:37.365 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:37.361+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:37.379 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:37.380 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:28:37.380 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:37.467 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:37.459+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:37.510 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.510 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.523 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:37.568 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:37.563+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:37.638 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.638 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.649 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:37.667 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:37.659+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:37.753 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.753 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.763 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:37.812 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:37.807+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:37.866 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.866 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.869 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:37.863+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:37.876 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:37.970 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:37.965+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:37.985 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:37.985 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:37.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:38.071 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:38.063+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:38.108 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:38.109 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:38.119 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:38.171 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:38.165+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:38.190 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:28:38.241 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:38.241 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:38.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:38.273 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:38.267+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:38.277 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:38.371 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:38.371 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:38.373 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:38.369+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:38.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:38.422 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:38.444 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:38.466 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:38.474 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:38.469+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:38.483 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:38.483 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:38.489 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:38.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:38.513 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:38.536 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:38.560 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:38.561 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:38.561 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:38.563 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:38.564 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:38.575 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:38.567+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:38.656 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 106.570945001, 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:38.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:38.661 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 87.347704410000006, 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:38.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:38.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.820491204000007, 2024-09-13T09:28:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.561244254999998, 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.561227700000003, 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.561212636, 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.560480873000003, 2024-09-13T09:28:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.560456967, 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.559764967, 2024-09-13T09:28:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.559750930999996, 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.559632553999997, 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.559439421999997, 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555504302000003, 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555495616000002, 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555486256000002, 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555477476999997, 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555468898000001, 2024-09-13T09:28:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555456214000003, 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.555447321999999, 2024-09-13T09:28:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:182783 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.989620446, 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99356371799999998, 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 182783 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663410+0000", 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663420+0000", 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0097e-05 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663440+0000", 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9559999999999999e-05 2024-09-13T09:28:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663448+0000", 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.824e-06 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663515+0000", 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6964999999999993e-05 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663527+0000", 2024-09-13T09:28:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2295e-05 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656700+0000", 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99317316300000003 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656806+0000", 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105889 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656976+0000", 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016978699999999999 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:38.756 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:38.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:38.758 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:38.753+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:38.858 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:38.851+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:38.870 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:38.870 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:38.880 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:38.959 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:38.951+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:38.987 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:38.987 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:38.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:39.060 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:39.053+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:39.104 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.677482961999999, 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:39.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.677467905999997, 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:39.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.677458498, 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.120 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.677444205, 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:39.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.677358499, 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:39.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.129 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.677002250000001, 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:39.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676955413999998, 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:39.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:39.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:39.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:39.138 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676916427999998, 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:39.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676896284000001, 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:39.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676877216000001, 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676835126999997, 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676721624999999, 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676703445999998, 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:39.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676693700999998, 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:39.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.170 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676683513999997, 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:39.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:39.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676590648000001, 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:39.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676487236, 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:39.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676165357999999, 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:39.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:39.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.188 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.675749416999999, 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:39.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:39.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:39.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.675495707000003, 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:39.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.198 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:39.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:39.200 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:39.193+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:39.300 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:39.293+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:39.306 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:39.306 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:39.306 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:39.306 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:39.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:39.402 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:39.395+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:39.420 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:39.421 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:39.421 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:39.421 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:39.421 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:39.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:39.503 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:39.495+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:39.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.576438233, 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:39.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:28:39.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 113.463218489, 2024-09-13T09:28:39.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:39.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:39.551 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 113.462820474, 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:28:39.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.556 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.787713598, 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.787698666, 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:28:39.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.78766555999999, 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:39.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.78764926700001, 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:28:39.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.787633152, 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.576 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:28:39.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.580 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.787602111, 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.787570488, 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:28:39.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:28:39.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.774622427, 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:28:39.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:39.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.594 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.77460537100001, 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.595 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:28:39.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:28:39.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.774589203, 2024-09-13T09:28:39.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:28:39.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.774375541, 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:28:39.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.77435623, 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:28:39.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.774339837, 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:28:39.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.774324175, 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.619 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:39.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.774297821, 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:28:39.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:28:39.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.245785093999999, 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.628 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:28:39.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:39.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:28:39.635 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 61.133870979000001, 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:39.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:39.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:39.640 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:39.642 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:39.642 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:39.639+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:39.743 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:39.735+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:39.760 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:39.760 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:39.770 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:39.844 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:39.841+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:39.874 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:39.874 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:39.875 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:39.875 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:39.885 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:39.945 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:39.941+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:39.999 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:39.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 95.257735515999997, 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:40.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:40.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:40.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.015562764999999, 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:40.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:40.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.01553955, 2024-09-13T09:28:40.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:40.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.011411232999997, 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:40.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:40.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:40.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.011358672999997, 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.021 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:40.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.011321977999998, 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.025 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:40.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.011302348000001, 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.030 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:40.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.011280169000003, 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:40.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:40.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.007524386, 2024-09-13T09:28:40.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:40.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:40.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.007499052, 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:40.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.048 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.007481214999999, 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:40.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.007468830000001, 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.053 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:40.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.007440068999998, 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.058 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:40.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:40.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.003287858, 2024-09-13T09:28:40.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:40.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.003259569999997, 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.003233436000002, 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.003208798999999, 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.076 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.003126225000003, 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:40.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.003083322000002, 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.085 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:40.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.9994576, 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.090 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:40.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.094 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:40.096 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:40.091+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:40.096 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:40.196 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:40.191+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:40.204 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:28:40.266 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:40.266 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:40.267 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:40.267 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:40.277 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:40.297 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:40.293+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:40.387 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:40.387 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:40.387 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:40.399 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:40.399 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:40.393+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:40.500 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:40.491+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:40.513 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 118.947119375, 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:40.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:40.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:40.519 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 86.359460995999996, 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:40.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.524 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 34.229168870999999, 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:40.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.644958420000002, 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.529 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:40.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.533 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243624787999998, 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.534 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:40.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.538 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.24360678, 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:40.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:40.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243589440999997, 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.543 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:40.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:40.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:40.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243572184000001, 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:40.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:40.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243554648, 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:40.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243537578000002, 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:40.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:40.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243523918999998, 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:40.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243514245999997, 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.568 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:40.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243499700999998, 2024-09-13T09:28:40.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243483034, 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:40.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:40.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.243465942999997, 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:40.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.586 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.241610367, 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:40.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:40.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:40.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.241594259000003, 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:40.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.595 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.241579895000001, 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:40.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:40.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:40.600 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.241566067000001, 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:40.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:40.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 33.241548367, 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:40.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:40.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:40.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:40.609 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:40.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:40.612 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:40.612 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:40.603+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:40.712 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:40.707+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:40.765 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:40.765 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:40.765 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:40.765 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:40.775 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:40.799 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:40.813 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:40.807+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:40.817 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:40.834 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:40.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:40.878 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:40.878 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:40.883 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:40.888 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:40.907 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:40.931 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:40.931 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:40.932 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:40.923+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:40.934 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:40.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:41.015 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:41.007+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:41.017 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.017 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.027 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:41.115 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:41.107+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:41.133 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.133 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.143 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:41.216 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:41.211+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:41.230 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:28:41.272 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.272 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:41.317 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:41.311+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:41.399 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.399 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.409 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:41.457 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:41.453+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:41.518 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:41.513+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.525 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:41.619 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:41.611+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:41.637 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.638 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:41.720 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:41.713+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.753 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:41.821 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:41.817+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:41.914 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:41.921 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:41.917+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:41.924 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:42.022 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:42.019+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:42.077 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.077 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:42.123 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:42.119+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:42.198 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.198 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.208 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:42.225 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:42.219+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:42.320 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.320 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.325 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:42.319+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:42.330 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:42.391 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0, 2] out_osds: [4] dead_osds: [4] live_osds: [0, 1, 2, 5, 6, 7, 3] 2024-09-13T09:28:42.391 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-13T09:28:42.391 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T09:28:42.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T09:28:42.397 INFO:tasks.ceph.osd.4:Started 2024-09-13T09:28:42.397 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:42.426 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:42.419+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:42.441 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.441 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:no valid command found; 10 closest matches: 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:0 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:1 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:2 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:abort 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:assert 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:config diff 2024-09-13T09:28:42.514 INFO:teuthology.orchestra.run.smithi099.stderr:config diff get 2024-09-13T09:28:42.515 INFO:teuthology.orchestra.run.smithi099.stderr:config get 2024-09-13T09:28:42.515 INFO:teuthology.orchestra.run.smithi099.stderr:config help [] 2024-09-13T09:28:42.515 INFO:teuthology.orchestra.run.smithi099.stderr:config set ... 2024-09-13T09:28:42.515 INFO:teuthology.orchestra.run.smithi099.stderr:admin_socket: invalid command 2024-09-13T09:28:42.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:28:42.516 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-13T09:28:42.527 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:42.521+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:42.559 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.559 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.569 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:42.628 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:42.621+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:42.685 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.685 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.695 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:42.729 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:42.725+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:42.798 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.798 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.808 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:42.830 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:42.825+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:42.915 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:42.915 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:42.925 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:42.931 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:42.923+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:42.935 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:43.011 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:43.005+0000 7f3dccb3f880 -1 Falling back to public interface 2024-09-13T09:28:43.032 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:43.027+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:43.033 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:43.044 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:43.081 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:43.100 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:43.119 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:43.133 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:43.127+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:43.139 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:43.148 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:43.148 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:43.148 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:43.148 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:43.148 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:43.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:43.162 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:43.193 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:43.234 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:43.227+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:43.235 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:43.236 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:43.238 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:43.239 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:43.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 121.716566153, 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.283 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:43.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:43.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.128907773999998, 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:43.288 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:43.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:43.293 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.998615649000001, 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:43.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:43.297 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.414405197999997, 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.298 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.302 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.013071566000001, 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:43.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.013053558000003, 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.307 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.311 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.013036219, 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:43.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:43.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.013018961999997, 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.316 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:43.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:43.320 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.013001426000002, 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:43.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.012984355999997, 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.325 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:43.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.329 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.012970697, 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:43.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:43.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.012961023999999, 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.335 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:43.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:43.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:43.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.012946479, 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:43.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:43.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:43.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.345 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.012929812000003, 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.346 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:43.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.012912720999999, 2024-09-13T09:28:43.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:43.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:43.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.011057145000002, 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:43.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.011041036999998, 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.011026673000003, 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.011012845000003, 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.372 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 36.010995145000003, 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:43.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:43.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.377 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:43.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:43.379 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:43.373+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:43.479 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:43.471+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:43.484 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:43.485 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:43.485 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:43.485 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:43.495 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:43.580 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:43.575+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:43.599 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:43.599 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:43.599 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:43.609 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:43.681 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:43.673+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:43.768 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:43.768 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:43.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:43.768 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:43.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.768 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187785458999997, 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:43.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187773221999997, 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.773 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:43.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187757824999998, 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:43.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187742950000001, 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.782 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:43.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187725741000001, 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187705286000003, 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:43.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:43.795 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187687206, 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:43.799 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187670830000002, 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:43.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187655479, 2024-09-13T09:28:43.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:43.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.808 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187638775000003, 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:43.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:43.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187620693, 2024-09-13T09:28:43.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187603948000003, 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.818 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:43.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:43.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.822 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.187559288999999, 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:43.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.184609848999997, 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.827 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:43.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.184589311000003, 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:43.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.184566695999997, 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.836 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:43.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.184547827999999, 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.184513234000001, 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:43.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.180234329000001, 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.850 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:43.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 56.180170357000002, 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:43.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:43.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:43.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:43.859 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:43.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:43.861 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:43.855+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:43.962 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:43.955+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:43.983 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:43.983 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:43.983 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:43.983 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:43.993 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:44.063 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:44.055+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:44.153 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:44.153 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:44.162 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:44.164 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:44.159+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:44.251 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:28:44.264 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:44.257+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:44.291 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:44.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:44.291 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.730452757000002, 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:44.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:44.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.296 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.102085183, 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.297 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:44.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.301 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.060506588999999, 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:44.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.303 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:44.305 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.059091410000001, 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.306 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:44.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:44.310 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.059072757000003, 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:44.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.314 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.059000208999997, 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:44.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:44.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.319 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058763433999999, 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058720602999998, 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.324 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:44.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058696941000001, 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:44.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058486909999999, 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.333 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.337 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058460039000003, 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:44.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058442145000001, 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.342 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:44.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:44.346 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058425898000003, 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:44.350 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058327834000004, 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.351 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.058094599, 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.356 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:44.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:44.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.057999991999999, 2024-09-13T09:28:44.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:44.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.057974467000001, 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.365 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:44.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.057951287999998, 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.057677265000002, 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.374 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:44.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 37.057493114000003, 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:44.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:44.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:44.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.383 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.383 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:44.383 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:44.383 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:44.384 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:44.385 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:44.377+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:44.485 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:44.479+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:44.501 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:44.501 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:44.502 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:44.502 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:44.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:44.587 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:44.579+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:44.619 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:44.620 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:44.630 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:44.688 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:44.681+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.651159788, 2024-09-13T09:28:44.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:44.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.427919196999994, 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:44.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:44.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:44.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 89.900705990999995, 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:44.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:44.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:44.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.641459042000001, 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:44.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.641442486999999, 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:44.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:44.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.641427423000003, 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:44.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:44.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.640695659999999, 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:44.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.640671754000003, 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:44.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.639979754000002, 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:44.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.639965717999999, 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.639847340999999, 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:44.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:44.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:44.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.639654209, 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.792 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:44.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635719088999998, 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.800 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635710402999997, 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:44.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:44.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635701042999997, 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.805 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:44.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:44.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635692263999999, 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:44.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:44.813 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635683685000004, 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:44.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635671000999999, 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:44.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:44.822 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.635662109000002, 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:44.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:182783 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.069835232999999, 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99356371799999998, 2024-09-13T09:28:44.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 182783 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663410+0000", 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663420+0000", 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0097e-05 2024-09-13T09:28:44.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663440+0000", 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9559999999999999e-05 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663448+0000", 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.824e-06 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663515+0000", 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6964999999999993e-05 2024-09-13T09:28:44.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663527+0000", 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2295e-05 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656700+0000", 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99317316300000003 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656806+0000", 2024-09-13T09:28:44.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105889 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656976+0000", 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016978699999999999 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:44.832 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:44.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:44.834 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:44.831+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:44.935 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:44.931+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:44.947 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:44.948 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:44.948 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:44.948 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:44.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:45.035 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:45.027+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:45.061 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:45.061 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:45.061 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:45.061 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:45.061 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:45.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:45.136 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:45.129+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:45.184 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755750364999997, 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.185 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:45.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755735309000002, 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:45.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:45.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755725900999998, 2024-09-13T09:28:45.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:45.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755711607999999, 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:45.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:45.204 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755625901999998, 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:45.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755269652999999, 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:45.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755222817000003, 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:45.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:45.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:45.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:45.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.221 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755183830999997, 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.227 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755163687, 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:45.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.232 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755144618999999, 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:45.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:45.237 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.755102530000002, 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:45.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:45.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754989027999997, 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:45.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754970849000003, 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.247 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:45.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754961104000003, 2024-09-13T09:28:45.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:45.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754950917000002, 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:45.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754858050999999, 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:45.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:45.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:45.265 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754754638999998, 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:45.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:45.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754432760999997, 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.754016819999997, 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:45.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:45.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.753763110000001, 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:45.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:45.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:45.283 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:45.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:45.285 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:45.279+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:45.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:45.386 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:45.379+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:45.407 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:45.407 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:45.407 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:45.407 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:45.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:45.438 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:45.465 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:45.485 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:45.486 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:45.479+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:45.509 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:45.519 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:45.519 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:45.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:45.532 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:45.553 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:45.576 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:45.576 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:45.577 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:45.581 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:45.581 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:45.586 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:45.579+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:45.630 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:45.631 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:45.640 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:45.687 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:45.679+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:45.759 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:45.759 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:45.770 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:45.788 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:45.781+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:45.882 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:45.882 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:45.889 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:45.883+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:45.892 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:45.991 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:45.983+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:45.997 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:45.997 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.007 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:46.091 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:46.083+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:46.114 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:46.114 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:46.192 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:46.185+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:46.263 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:28:46.293 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:46.287+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:46.301 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:46.301 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.315 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:46.394 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:46.387+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:46.493 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:46.493 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.494 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:46.487+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:46.502 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:46.595 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:46.589+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:46.652 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:46.652 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.653 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:46.649+0000 7f3dccb3f880 -1 osd.4 142 log_to_monitors true 2024-09-13T09:28:46.662 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:46.696 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:46.689+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:46.768 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:46.768 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.777 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:46.797 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:46.791+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:46.888 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:46.888 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:46.898 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:46.898 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:46.893+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:46.999 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:46.991+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:47.052 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.052 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.062 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:47.100 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:47.095+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:47.166 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.167 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.176 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:47.201 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:47.197+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:47.280 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:28:47.300 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:47.293+0000 7f3dc042d700 -1 osd.4 172 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:28:47.302 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:47.295+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:47.346 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.346 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:47.403 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:47.397+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:47.483 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.483 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:47.504 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:47.495+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:47.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:47.582 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:47.596 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.596 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.604 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:47.597+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:47.606 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:47.627 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:47.628 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:47.638 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:28:47.705 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:47.699+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:47.706 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.706 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:47.731 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:47.751 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:47.789 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:47.806 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:47.801+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:47.813 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:47.822 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:47.823 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:47.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:47.837 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:47.861 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:47.884 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:47.885 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:47.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:47.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:47.907 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:47.899+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:47.983 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:47.983 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:47.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:48.008 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:48.003+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674694141000003, 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:48.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:48.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.107 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674679085000001, 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.108 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:48.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.112 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674669676999997, 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:48.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:48.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674655383999998, 2024-09-13T09:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:48.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.121 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674569677999997, 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.122 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:48.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:48.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674213428999998, 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:48.130 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674166593000002, 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:48.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:48.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674127607000003, 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.136 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:48.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:48.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674107462999999, 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:48.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:48.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674088394999998, 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:48.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:48.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:48.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.674046306000001, 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673932804000003, 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:48.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.159 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673914625000002, 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:48.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673904880000002, 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.164 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.168 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673894693000001, 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:48.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673801826999998, 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.173 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:48.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673698414999997, 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:48.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:48.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.673376537000003, 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.182 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:48.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.672960596000003, 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:48.186 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:48.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.672706886, 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:48.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:48.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.195 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:48.197 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:48.197 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:48.189+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:48.284 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:28:48.297 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:48.291+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:48.298 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:28:48.368 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:48.368 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:48.368 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:48.368 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:48.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:48.398 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:48.393+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:48.497 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:48.497 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:48.497 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:48.497 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:48.497 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:48.498 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:48.493+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:48.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:48.599 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:48.593+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 185.701284206, 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:28:48.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:28:48.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.58806446200001, 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:28:48.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:28:48.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:28:48.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.587666447, 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.676 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:48.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:28:48.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.912559571, 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:28:48.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.912544639, 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:48.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.912511533, 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:28:48.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.91249524, 2024-09-13T09:28:48.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.698 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.912479125, 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:28:48.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:28:48.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:28:48.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:48.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.912448084, 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:28:48.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:28:48.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.91241646100001, 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.708 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:28:48.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.8994684, 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.717 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.899451344, 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:48.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:28:48.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.899435176, 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:28:48.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.726 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.899221514, 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:28:48.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.899202203, 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:48.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:28:48.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:28:48.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.89918581000001, 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:28:48.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.899170148, 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:28:48.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:48.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:48.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 121.899143794, 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:48.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:28:48.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.370631067000005, 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:28:48.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.258716952, 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:48.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:28:48.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.765 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:48.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:48.766 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:48.767 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:48.768 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:48.761+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:48.868 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:48.861+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:48.883 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:48.883 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:48.884 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:48.893 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:48.969 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:48.965+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:48.998 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:48.999 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:48.999 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:48.999 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:48.999 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:49.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:49.070 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:49.063+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:49.117 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:49.117 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:49.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 104.375862252, 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.118 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:49.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:49.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.133689500999999, 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:49.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.133666286, 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:49.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:49.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:49.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.129537968999998, 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:49.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.129485408999997, 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.140 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:49.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:49.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.145 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.129448713999999, 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:49.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.129429084000002, 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:49.150 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:49.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:49.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:49.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.129406905000003, 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.156 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:49.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.157 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:49.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:49.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:49.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:49.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.125651122000001, 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:49.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:49.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.125625788000001, 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.171 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:49.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:49.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:49.176 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.176 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.125607950999999, 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:49.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:49.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:49.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.125595566000001, 2024-09-13T09:28:49.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.185 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.125566804999998, 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.186 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:49.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.121414594000001, 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.191 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.197 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.197 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:49.197 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.121386305999998, 2024-09-13T09:28:49.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:49.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:49.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.202 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.121360172000003, 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.203 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:49.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.121335535, 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.209 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:49.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:49.213 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.121252961000003, 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:49.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:49.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:49.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:49.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:49.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:49.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:49.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.219 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.220 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.220 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.220 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:49.220 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.121210058000003, 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:49.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:49.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:49.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.226 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.117584336, 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:49.227 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:49.231 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:49.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:49.233 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:49.225+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:49.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:28:49.306 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:28:49.319 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:28:49.333 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:49.329+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:49.353 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:49.354 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:49.354 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:49.354 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:49.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:49.434 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:49.429+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:49.469 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:49.469 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:49.469 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:49.469 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:49.469 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:49.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:49.535 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:49.527+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:49.633 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:49.633 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:49.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:49.633 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [] 2024-09-13T09:28:49.634 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:49.636 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:49.629+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:49.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:49.737 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:49.731+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:49.749 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:49.750 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:49.750 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:49.750 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:49.759 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:49.839 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:49.831+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:49.864 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:49.864 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:49.865 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:49.865 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:49.865 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:49.874 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:49.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:49.940 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:49.937+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:50.030 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:50.030 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:50.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 128.46381359700001, 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:50.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:50.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:50.036 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 95.876155217999994, 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:50.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:50.042 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.745863092999997, 2024-09-13T09:28:50.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:50.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:50.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:50.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:50.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:50.051 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 43.161652642, 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:50.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:50.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:50.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:50.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:50.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:50.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:50.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:50.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.060 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760319010000003, 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:50.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760301001999999, 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:50.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:50.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760283663000003, 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:50.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:50.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.075 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760266406, 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:50.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:50.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:50.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:50.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760248869999998, 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:50.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.7602318, 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:50.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760218141000003, 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.092 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:50.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760208468000002, 2024-09-13T09:28:50.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:50.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760193923000003, 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.101 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:50.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:50.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760177255999999, 2024-09-13T09:28:50.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:50.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.110 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.760160165000002, 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:50.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:50.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.758304588999998, 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:50.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.758288481000001, 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:50.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.758274116999999, 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:50.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:50.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:50.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.758260288999999, 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:50.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:50.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:50.133 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 42.758242588999998, 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:50.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:50.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:50.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:50.138 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:50.139 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:50.140 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:50.140 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:50.140 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:50.140 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:50.140 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:50.141 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:50.131+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:50.160 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:50.185 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:28:50.186 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:28:50.186 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:50.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:28:50.218 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:50.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-13T09:28:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:50.222 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:50.223 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:50.280 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:50.275+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:50.281 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:50.281 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:50.293 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:50.340 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:50.333+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:50.402 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:50.403 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:50.413 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:50.441 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:50.437+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:50.527 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:50.527 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:50.537 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:50.542 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:50.535+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:50.643 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:50.643 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:50.644 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:50.637+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:50.653 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:50.745 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:50.739+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:50.760 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:50.760 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:50.770 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:50.845 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:50.837+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:50.925 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:50.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:50.946 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:50.939+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:51.047 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:51.043+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:51.050 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.050 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:51.148 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:51.141+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:51.162 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.162 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:51.249 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:51.247+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:51.278 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.278 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:51.314 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:28:51.330 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:28:51.372 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:51.367+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:51.399 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.399 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.409 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:51.452 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:51.445+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:51.516 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.516 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:51.553 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:51.545+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:51.628 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.629 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.638 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:51.682 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:51.677+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:51.754 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:51.747+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:51.793 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.793 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:51.856 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:51.849+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:51.915 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:51.915 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:51.925 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:51.969 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:51.965+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:52.032 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:52.032 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:52.042 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:52.057 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:52.053+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:52.158 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:52.157+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:52.162 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:52.162 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:52.172 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:52.205 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:28:52.237 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:52.260 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:52.257+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:52.284 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:52.284 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:52.294 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:52.326 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:28:52.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:28:52.360 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:52.353+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:52.412 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:52.434 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:52.436 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:52.436 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:52.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:52.460 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:52.500 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:52.501 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:52.497+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:52.524 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:52.560 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:52.560 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:52.560 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:52.561 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:52.555+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:52.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:52.590 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:52.611 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:52.614 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:52.661 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:52.653+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:52.674 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:52.674 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:52.675 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:52.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:52.762 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:52.757+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:52.792 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:52.792 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:52.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:52.792 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [] 2024-09-13T09:28:52.792 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:52.802 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:52.863 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:52.855+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:52.921 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:52.921 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:52.922 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:52.922 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:52.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:52.963 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:52.957+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:53.045 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:53.045 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:53.045 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:53.045 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:53.045 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:53.051 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:53.096 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:53.091+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:53.165 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:53.165 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 131.59913207700001, 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.166 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:53.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:53.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 99.011473698000003, 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:53.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.176 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 46.881181572999999, 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:53.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:53.181 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 46.296971122000002, 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895637489999999, 2024-09-13T09:28:53.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:53.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:53.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895619482000001, 2024-09-13T09:28:53.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:53.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:53.196 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895602142999998, 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:53.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:53.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.201 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895584886000002, 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.202 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:53.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:53.206 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.89556735, 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:53.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:53.211 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895550280000002, 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.212 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:53.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895536620999998, 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:53.216 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:53.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.220 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895526947999997, 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.221 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:53.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895512402999998, 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:53.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895495736000001, 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:53.230 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:53.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:53.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.895478644999997, 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:53.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:53.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.239 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.893623069, 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:53.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.893606961000003, 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:53.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.249 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.893592597000001, 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:53.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.893578769000001, 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:53.254 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:53.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.258 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 45.893561069, 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:53.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.263 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:53.264 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:53.159+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:53.265 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:53.266 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:53.263+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:53.337 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:28:53.338 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:28:53.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:28:53.400 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:53.395+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:188644 3.6 e178/60 3:6797c975:::rbd_header.1118f0c8e51e:head v 178'866, mlcod=178'864)", 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:53.401553+0000", 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.001609524, 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001629828, 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.404 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401553+0000", 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401553+0000", 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401552+0000", 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401562+0000", 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0560999999999999e-05 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401581+0000", 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8881000000000001e-05 2024-09-13T09:28:53.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401590+0000", 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0880000000000001e-06 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401671+0000", 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0519000000000006e-05 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.407 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:53.401715+0000", 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3779000000000002e-05 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 1 2024-09-13T09:28:53.408 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:53.415 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:53.467 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:53.459+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:53.542 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:53.542 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:53.543 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:53.543 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:53.543 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:53.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:53.568 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:53.561+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:53.701 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:53.691+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:53.800 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:53.795+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230859565000003, 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.811 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.814 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.815 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230847327999996, 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.816 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230831930999997, 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.820 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.823 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.824 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230817056000006, 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:28:53.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.829 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230799847, 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:28:53.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:28:53.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.834 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230779392000002, 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230761311999998, 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.839 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:28:53.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.843 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230744935999994, 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:28:53.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230729585000006, 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.848 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:28:53.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230712881000002, 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.853 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:28:53.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230694799000005, 2024-09-13T09:28:53.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:28:53.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.862 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.862 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:53.864 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230678053999995, 2024-09-13T09:28:53.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:28:53.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:28:53.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.230633394999998, 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:28:53.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.227683955000003, 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:28:53.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.876 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.227663417000002, 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.878 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:28:53.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.227640801999996, 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.885 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:28:53.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.227621933999998, 2024-09-13T09:28:53.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:53.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:28:53.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:28:53.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.227587339999999, 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.892 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:28:53.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:28:53.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.223308435000007, 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.897 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:28:53.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:28:53.901 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 66.223244463, 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:53.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:53.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.906 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.906 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:53.906 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:53.906 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:53.907 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:53.908 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:53.903+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:54.008 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:54.005+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:54.065 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:54.066 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:54.066 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:54.066 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:54.075 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:54.109 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:54.105+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:54.186 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:54.186 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:54.186 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:54.186 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:54.186 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:54.196 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:54.240 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:54.235+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.745403500000002, 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:28:54.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:28:54.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.117035926, 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.313 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:54.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:28:54.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.075457331999999, 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:28:54.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:28:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:54.322 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.074042153000001, 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:28:54.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.074023500000003, 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:28:54.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:28:54.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.331 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073950951999997, 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.332 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073714176999999, 2024-09-13T09:28:54.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:54.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:28:54.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.341 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073671345999998, 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:28:54.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:28:54.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073647684000001, 2024-09-13T09:28:54.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:28:54.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073437652999999, 2024-09-13T09:28:54.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073410782000003, 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.356 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:28:54.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:28:54.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073392888000001, 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:28:54.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073376641000003, 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.365 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:28:54.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.369 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073278577000004, 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:28:54.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.073045342, 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:28:54.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:28:54.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:28:54.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.380 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.072950734999999, 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:28:54.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:28:54.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:28:54.385 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.072925210000001, 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.386 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.390 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.072902030999998, 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:28:54.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.072628008000002, 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:28:54.395 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:28:54.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.399 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 47.072443857000003, 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:28:54.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:54.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:28:54.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:28:54.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:54.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:54.405 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:54.407 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:28:54.408 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:28:54.408 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:28:54.408 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:28:54.409 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:54.401+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:54.507 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:54.499+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:54.529 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:54.530 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:54.530 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:54.530 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:54.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:28:54.609 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:54.601+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:54.625 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:54.641 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:54.641 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:54.641 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:54.641 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:54.642 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:54.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:28:54.709 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:54.705+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:54.729 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:28:54.730 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:28:54.744 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-13T09:28:54.569601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555986987,"num_objects":490,"num_object_clones":73,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":93,"num_read":247692,"num_read_kb":750737,"num_write":30890,"num_write_kb":441296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6793,"num_bytes_recovered":385387821,"num_keys_recovered":1247,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24456,"ondisk_log_size":24456,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":415384,"kb_used_data":30896,"kb_used_omap":23,"kb_used_meta":384360,"kb_avail":660187496,"statfs":{"total":676457349120,"available":676031995904,"internally_reserved":0,"allocated":31637504,"data_stored":120280205,"data_compressed":6829803,"data_compressed_allocated":9342976,"data_compressed_original":106519000,"omap_allocated":24352,"internal_metadata":393584864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-188557283,"num_objects":-191,"num_object_clones":-40,"num_object_copies":-382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-191,"num_whiteouts":-23,"num_read":1279,"num_read_kb":1097,"num_write":305,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001181"},"pg_stats":[{"pgid":"3.1f","version":"176'517","reported_seq":2078,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.314771+0000","last_change":"2024-09-13T09:28:51.315176+0000","last_active":"2024-09-13T09:28:51.314771+0000","last_peered":"2024-09-13T09:28:51.314771+0000","last_clean":"2024-09-13T09:28:51.314771+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:28:51.314771+0000","last_undegraded":"2024-09-13T09:28:51.314771+0000","last_fullsized":"2024-09-13T09:28:51.314771+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0097452289999999994,"stat_sum":{"num_bytes":12223234,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3176,"num_read_kb":16241,"num_write":511,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1e","version":"175'813","reported_seq":5689,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.397717+0000","last_change":"2024-09-13T09:28:50.202702+0000","last_active":"2024-09-13T09:28:50.397717+0000","last_peered":"2024-09-13T09:28:50.397717+0000","last_clean":"2024-09-13T09:28:50.397717+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:28:50.397717+0000","last_undegraded":"2024-09-13T09:28:50.397717+0000","last_fullsized":"2024-09-13T09:28:50.397717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01203196,"stat_sum":{"num_bytes":8497000,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":6798,"num_read_kb":14454,"num_write":1116,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.1d","version":"176'618","reported_seq":5377,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.326769+0000","last_change":"2024-09-13T09:28:51.327075+0000","last_active":"2024-09-13T09:28:51.326769+0000","last_peered":"2024-09-13T09:28:51.326769+0000","last_clean":"2024-09-13T09:28:51.326769+0000","last_became_active":"2024-09-13T09:28:12.655560+0000","last_became_peered":"2024-09-13T09:28:12.655560+0000","last_unstale":"2024-09-13T09:28:51.326769+0000","last_undegraded":"2024-09-13T09:28:51.326769+0000","last_fullsized":"2024-09-13T09:28:51.326769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:17.740965+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010390780000000001,"stat_sum":{"num_bytes":19666007,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":5233,"num_read_kb":24957,"num_write":713,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":21645529,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1c","version":"176'772","reported_seq":6070,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.385882+0000","last_change":"2024-09-13T09:28:51.317727+0000","last_active":"2024-09-13T09:28:51.385882+0000","last_peered":"2024-09-13T09:28:51.385882+0000","last_clean":"2024-09-13T09:28:51.385882+0000","last_became_active":"2024-09-13T09:28:12.654993+0000","last_became_peered":"2024-09-13T09:28:12.654993+0000","last_unstale":"2024-09-13T09:28:51.385882+0000","last_undegraded":"2024-09-13T09:28:51.385882+0000","last_fullsized":"2024-09-13T09:28:51.385882+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:26.519049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001236661,"stat_sum":{"num_bytes":17355159,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":6207,"num_read_kb":19081,"num_write":1075,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":24632032,"num_keys_recovered":228,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1b","version":"175'673","reported_seq":6763,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.307534+0000","last_change":"2024-09-13T09:28:51.309071+0000","last_active":"2024-09-13T09:28:51.307534+0000","last_peered":"2024-09-13T09:28:51.307534+0000","last_clean":"2024-09-13T09:28:51.307534+0000","last_became_active":"2024-09-13T09:28:11.652761+0000","last_became_peered":"2024-09-13T09:28:11.652761+0000","last_unstale":"2024-09-13T09:28:51.307534+0000","last_undegraded":"2024-09-13T09:28:51.307534+0000","last_fullsized":"2024-09-13T09:28:51.307534+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:09.566533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039085379999999996,"stat_sum":{"num_bytes":3750401,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":8005,"num_read_kb":16391,"num_write":692,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1a","version":"172'619","reported_seq":13078,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.511347+0000","last_change":"2024-09-13T09:28:51.317609+0000","last_active":"2024-09-13T09:28:51.511347+0000","last_peered":"2024-09-13T09:28:51.511347+0000","last_clean":"2024-09-13T09:28:51.511347+0000","last_became_active":"2024-09-13T09:28:38.183181+0000","last_became_peered":"2024-09-13T09:28:38.183181+0000","last_unstale":"2024-09-13T09:28:51.511347+0000","last_undegraded":"2024-09-13T09:28:51.511347+0000","last_fullsized":"2024-09-13T09:28:51.511347+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:13:08.070603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013251249999999999,"stat_sum":{"num_bytes":34090509,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":13366,"num_read_kb":23715,"num_write":697,"num_write_kb":9206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":22514520,"num_keys_recovered":115,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.19","version":"175'633","reported_seq":3650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.316215+0000","last_change":"2024-09-13T09:28:51.316651+0000","last_active":"2024-09-13T09:28:51.316215+0000","last_peered":"2024-09-13T09:28:51.316215+0000","last_clean":"2024-09-13T09:28:51.316215+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:28:51.316215+0000","last_undegraded":"2024-09-13T09:28:51.316215+0000","last_fullsized":"2024-09-13T09:28:51.316215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055758900000000004,"stat_sum":{"num_bytes":20467685,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":4338,"num_read_kb":14826,"num_write":615,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.18","version":"175'634","reported_seq":5517,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.185653+0000","last_change":"2024-09-13T09:28:50.185653+0000","last_active":"2024-09-13T09:28:50.185653+0000","last_peered":"2024-09-13T09:28:50.185653+0000","last_clean":"2024-09-13T09:28:50.185653+0000","last_became_active":"2024-09-13T09:28:37.183228+0000","last_became_peered":"2024-09-13T09:28:37.183228+0000","last_unstale":"2024-09-13T09:28:50.185653+0000","last_undegraded":"2024-09-13T09:28:50.185653+0000","last_fullsized":"2024-09-13T09:28:50.185653+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:34.997664+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0081704910000000002,"stat_sum":{"num_bytes":7771224,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":7146,"num_read_kb":22085,"num_write":609,"num_write_kb":15608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.17","version":"172'760","reported_seq":2569,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.310590+0000","last_change":"2024-09-13T09:28:51.310947+0000","last_active":"2024-09-13T09:28:51.310590+0000","last_peered":"2024-09-13T09:28:51.310590+0000","last_clean":"2024-09-13T09:28:51.310590+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:28:51.310590+0000","last_undegraded":"2024-09-13T09:28:51.310590+0000","last_fullsized":"2024-09-13T09:28:51.310590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:47.404587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053277510000000004,"stat_sum":{"num_bytes":2255815,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":4228,"num_read_kb":9889,"num_write":846,"num_write_kb":4773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.16","version":"173'581","reported_seq":6059,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.317881+0000","last_change":"2024-09-13T09:28:51.318342+0000","last_active":"2024-09-13T09:28:51.317881+0000","last_peered":"2024-09-13T09:28:51.317881+0000","last_clean":"2024-09-13T09:28:51.317881+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:28:51.317881+0000","last_undegraded":"2024-09-13T09:28:51.317881+0000","last_fullsized":"2024-09-13T09:28:51.317881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015313200000000001,"stat_sum":{"num_bytes":24461347,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5787,"num_read_kb":36038,"num_write":669,"num_write_kb":20370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.15","version":"176'486","reported_seq":10460,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.489901+0000","last_change":"2024-09-13T09:28:51.321372+0000","last_active":"2024-09-13T09:28:51.489901+0000","last_peered":"2024-09-13T09:28:51.489901+0000","last_clean":"2024-09-13T09:28:51.489901+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:28:51.489901+0000","last_undegraded":"2024-09-13T09:28:51.489901+0000","last_fullsized":"2024-09-13T09:28:51.489901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010612369,"stat_sum":{"num_bytes":7628313,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":10262,"num_read_kb":33263,"num_write":488,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.14","version":"176'599","reported_seq":6388,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.326498+0000","last_change":"2024-09-13T09:28:51.326733+0000","last_active":"2024-09-13T09:28:51.326498+0000","last_peered":"2024-09-13T09:28:51.326498+0000","last_clean":"2024-09-13T09:28:51.326498+0000","last_became_active":"2024-09-13T09:28:11.648374+0000","last_became_peered":"2024-09-13T09:28:11.648374+0000","last_unstale":"2024-09-13T09:28:51.326498+0000","last_undegraded":"2024-09-13T09:28:51.326498+0000","last_fullsized":"2024-09-13T09:28:51.326498+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:26.377449+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015744873,"stat_sum":{"num_bytes":23863900,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":6502,"num_read_kb":13665,"num_write":644,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":8018413,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.7","version":"0'0","reported_seq":186,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.177564+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:28:50.177564+0000","last_peered":"2024-09-13T09:28:50.177564+0000","last_clean":"2024-09-13T09:28:50.177564+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:28:50.177564+0000","last_undegraded":"2024-09-13T09:28:50.177564+0000","last_fullsized":"2024-09-13T09:28:50.177564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"175'520","reported_seq":2788,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.318632+0000","last_change":"2024-09-13T09:28:51.319044+0000","last_active":"2024-09-13T09:28:51.318632+0000","last_peered":"2024-09-13T09:28:51.318632+0000","last_clean":"2024-09-13T09:28:51.318632+0000","last_became_active":"2024-09-13T09:27:05.753131+0000","last_became_peered":"2024-09-13T09:27:05.753131+0000","last_unstale":"2024-09-13T09:28:51.318632+0000","last_undegraded":"2024-09-13T09:28:51.318632+0000","last_fullsized":"2024-09-13T09:28:51.318632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:21.776084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021241599999999999,"stat_sum":{"num_bytes":32070734,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":2305,"num_read_kb":6499,"num_write":566,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.6","version":"0'0","reported_seq":194,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.177843+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:28:50.177843+0000","last_peered":"2024-09-13T09:28:50.177843+0000","last_clean":"2024-09-13T09:28:50.177843+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:28:50.177843+0000","last_undegraded":"2024-09-13T09:28:50.177843+0000","last_fullsized":"2024-09-13T09:28:50.177843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"175'902","reported_seq":4421,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.316966+0000","last_change":"2024-09-13T09:28:51.317252+0000","last_active":"2024-09-13T09:28:51.316966+0000","last_peered":"2024-09-13T09:28:51.316966+0000","last_clean":"2024-09-13T09:28:51.316966+0000","last_became_active":"2024-09-13T09:28:39.186243+0000","last_became_peered":"2024-09-13T09:28:39.186243+0000","last_unstale":"2024-09-13T09:28:51.316966+0000","last_undegraded":"2024-09-13T09:28:51.316966+0000","last_fullsized":"2024-09-13T09:28:51.316966+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":902,"log_dups_size":0,"ondisk_log_size":902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:40.874996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060269,"stat_sum":{"num_bytes":5584529,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":5847,"num_read_kb":19431,"num_write":1182,"num_write_kb":22716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-13T09:28:49.297227+0000","last_change":"2024-09-13T09:28:33.240144+0000","last_active":"2024-09-13T09:28:49.297227+0000","last_peered":"2024-09-13T09:28:49.297227+0000","last_clean":"2024-09-13T09:28:49.297227+0000","last_became_active":"2024-09-13T09:28:33.239677+0000","last_became_peered":"2024-09-13T09:28:33.239677+0000","last_unstale":"2024-09-13T09:28:49.297227+0000","last_undegraded":"2024-09-13T09:28:49.297227+0000","last_fullsized":"2024-09-13T09:28:49.297227+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:04:26.140186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"175'867","reported_seq":9056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.201921+0000","last_change":"2024-09-13T09:28:50.202321+0000","last_active":"2024-09-13T09:28:50.201921+0000","last_peered":"2024-09-13T09:28:50.201921+0000","last_clean":"2024-09-13T09:28:50.201921+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:28:50.201921+0000","last_undegraded":"2024-09-13T09:28:50.201921+0000","last_fullsized":"2024-09-13T09:28:50.201921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011828231999999999,"stat_sum":{"num_bytes":23403538,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1122,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":130,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-13T09:28:49.296161+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:28:49.296161+0000","last_peered":"2024-09-13T09:28:49.296161+0000","last_clean":"2024-09-13T09:28:49.296161+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:28:49.296161+0000","last_undegraded":"2024-09-13T09:28:49.296161+0000","last_fullsized":"2024-09-13T09:28:49.296161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'845","reported_seq":5884,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.498176+0000","last_change":"2024-09-13T09:28:51.309932+0000","last_active":"2024-09-13T09:28:51.498176+0000","last_peered":"2024-09-13T09:28:51.498176+0000","last_clean":"2024-09-13T09:28:51.498176+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:28:51.498176+0000","last_undegraded":"2024-09-13T09:28:51.498176+0000","last_fullsized":"2024-09-13T09:28:51.498176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046189259999999998,"stat_sum":{"num_bytes":5288604,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":7912,"num_read_kb":24806,"num_write":998,"num_write_kb":10476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1","version":"176'1366","reported_seq":3870,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.543257+0000","last_change":"2024-09-13T09:28:51.306517+0000","last_active":"2024-09-13T09:28:51.543257+0000","last_peered":"2024-09-13T09:28:51.543257+0000","last_clean":"2024-09-13T09:28:51.543257+0000","last_became_active":"2024-09-13T09:28:12.650128+0000","last_became_peered":"2024-09-13T09:28:12.650128+0000","last_unstale":"2024-09-13T09:28:51.543257+0000","last_undegraded":"2024-09-13T09:28:51.543257+0000","last_fullsized":"2024-09-13T09:28:51.543257+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1366,"log_dups_size":0,"ondisk_log_size":1366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:23.557202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514903,"stat_sum":{"num_bytes":1200226,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":7691,"num_read_kb":20013,"num_write":2047,"num_write_kb":7019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":10575408,"num_keys_recovered":140,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.0","version":"19'117","reported_seq":316,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.188706+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:28:50.188706+0000","last_peered":"2024-09-13T09:28:50.188706+0000","last_clean":"2024-09-13T09:28:50.188706+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:28:50.188706+0000","last_undegraded":"2024-09-13T09:28:50.188706+0000","last_fullsized":"2024-09-13T09:28:50.188706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":174,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.190791+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:28:50.190791+0000","last_peered":"2024-09-13T09:28:50.190791+0000","last_clean":"2024-09-13T09:28:50.190791+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:28:50.190791+0000","last_undegraded":"2024-09-13T09:28:50.190791+0000","last_fullsized":"2024-09-13T09:28:50.190791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"175'985","reported_seq":9287,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.317405+0000","last_change":"2024-09-13T09:28:51.317679+0000","last_active":"2024-09-13T09:28:51.317405+0000","last_peered":"2024-09-13T09:28:51.317405+0000","last_clean":"2024-09-13T09:28:51.317405+0000","last_became_active":"2024-09-13T09:28:39.186937+0000","last_became_peered":"2024-09-13T09:28:39.186937+0000","last_unstale":"2024-09-13T09:28:51.317405+0000","last_undegraded":"2024-09-13T09:28:51.317405+0000","last_fullsized":"2024-09-13T09:28:51.317405+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:51.885937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095999300000000002,"stat_sum":{"num_bytes":23323589,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":10796,"num_read_kb":23352,"num_write":1403,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.2","version":"19'2","reported_seq":198,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.188973+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:28:50.188973+0000","last_peered":"2024-09-13T09:28:50.188973+0000","last_clean":"2024-09-13T09:28:50.188973+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:28:50.188973+0000","last_undegraded":"2024-09-13T09:28:50.188973+0000","last_fullsized":"2024-09-13T09:28:50.188973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'765","reported_seq":11108,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.500712+0000","last_change":"2024-09-13T09:28:51.310518+0000","last_active":"2024-09-13T09:28:51.500712+0000","last_peered":"2024-09-13T09:28:51.500712+0000","last_clean":"2024-09-13T09:28:51.500712+0000","last_became_active":"2024-09-13T09:28:12.654541+0000","last_became_peered":"2024-09-13T09:28:12.654541+0000","last_unstale":"2024-09-13T09:28:51.500712+0000","last_undegraded":"2024-09-13T09:28:51.500712+0000","last_fullsized":"2024-09-13T09:28:51.500712+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:32.252069+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0056077749999999997,"stat_sum":{"num_bytes":38321324,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":10021,"num_read_kb":38964,"num_write":855,"num_write_kb":31778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":52739384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.1","version":"0'0","reported_seq":190,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.189048+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:28:50.189048+0000","last_peered":"2024-09-13T09:28:50.189048+0000","last_clean":"2024-09-13T09:28:50.189048+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:28:50.189048+0000","last_undegraded":"2024-09-13T09:28:50.189048+0000","last_fullsized":"2024-09-13T09:28:50.189048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'2093","reported_seq":6932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.366692+0000","last_change":"2024-09-13T09:28:50.206159+0000","last_active":"2024-09-13T09:28:50.366692+0000","last_peered":"2024-09-13T09:28:50.366692+0000","last_clean":"2024-09-13T09:28:50.366692+0000","last_became_active":"2024-09-13T09:27:05.744803+0000","last_became_peered":"2024-09-13T09:27:05.744803+0000","last_unstale":"2024-09-13T09:28:50.366692+0000","last_undegraded":"2024-09-13T09:28:50.366692+0000","last_fullsized":"2024-09-13T09:28:50.366692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2093,"log_dups_size":0,"ondisk_log_size":2093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:40.185419+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015829551000000001,"stat_sum":{"num_bytes":6636818,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":14692,"num_read_kb":17088,"num_write":3712,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":188,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.188801+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:28:50.188801+0000","last_peered":"2024-09-13T09:28:50.188801+0000","last_clean":"2024-09-13T09:28:50.188801+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:28:50.188801+0000","last_undegraded":"2024-09-13T09:28:50.188801+0000","last_fullsized":"2024-09-13T09:28:50.188801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'625","reported_seq":3350,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.191019+0000","last_change":"2024-09-13T09:28:50.191589+0000","last_active":"2024-09-13T09:28:50.191019+0000","last_peered":"2024-09-13T09:28:50.191019+0000","last_clean":"2024-09-13T09:28:50.191019+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:28:50.191019+0000","last_undegraded":"2024-09-13T09:28:50.191019+0000","last_fullsized":"2024-09-13T09:28:50.191019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011978959999999999,"stat_sum":{"num_bytes":8521299,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":4377,"num_read_kb":15143,"num_write":647,"num_write_kb":7476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.9","version":"172'464","reported_seq":11900,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.177751+0000","last_change":"2024-09-13T09:28:50.178009+0000","last_active":"2024-09-13T09:28:50.177751+0000","last_peered":"2024-09-13T09:28:50.177751+0000","last_clean":"2024-09-13T09:28:50.177751+0000","last_became_active":"2024-09-13T09:28:36.178628+0000","last_became_peered":"2024-09-13T09:28:36.178628+0000","last_unstale":"2024-09-13T09:28:50.177751+0000","last_undegraded":"2024-09-13T09:28:50.177751+0000","last_fullsized":"2024-09-13T09:28:50.177751+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:16.388720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052350499999999996,"stat_sum":{"num_bytes":30532891,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":10498,"num_read_kb":27484,"num_write":476,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":25814267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.a","version":"176'557","reported_seq":7214,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.505597+0000","last_change":"2024-09-13T09:28:51.321343+0000","last_active":"2024-09-13T09:28:51.505597+0000","last_peered":"2024-09-13T09:28:51.505597+0000","last_clean":"2024-09-13T09:28:51.505597+0000","last_became_active":"2024-09-13T09:28:11.646271+0000","last_became_peered":"2024-09-13T09:28:11.646271+0000","last_unstale":"2024-09-13T09:28:51.505597+0000","last_undegraded":"2024-09-13T09:28:51.505597+0000","last_fullsized":"2024-09-13T09:28:51.505597+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:49.883994+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011730127999999999,"stat_sum":{"num_bytes":5582864,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":6480,"num_read_kb":26569,"num_write":533,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":13216032,"num_keys_recovered":88,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.b","version":"174'592","reported_seq":4353,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.202766+0000","last_change":"2024-09-13T09:28:50.203011+0000","last_active":"2024-09-13T09:28:50.202766+0000","last_peered":"2024-09-13T09:28:50.202766+0000","last_clean":"2024-09-13T09:28:50.202766+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:28:50.202766+0000","last_undegraded":"2024-09-13T09:28:50.202766+0000","last_fullsized":"2024-09-13T09:28:50.202766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010154699999999999,"stat_sum":{"num_bytes":57229243,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":3,"num_read":3847,"num_read_kb":28423,"num_write":668,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.c","version":"175'571","reported_seq":7529,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.316112+0000","last_change":"2024-09-13T09:28:51.316907+0000","last_active":"2024-09-13T09:28:51.316112+0000","last_peered":"2024-09-13T09:28:51.316112+0000","last_clean":"2024-09-13T09:28:51.316112+0000","last_became_active":"2024-09-13T09:27:05.745595+0000","last_became_peered":"2024-09-13T09:27:05.745595+0000","last_unstale":"2024-09-13T09:28:51.316112+0000","last_undegraded":"2024-09-13T09:28:51.316112+0000","last_fullsized":"2024-09-13T09:28:51.316112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:38.773883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00101932,"stat_sum":{"num_bytes":21135992,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":7720,"num_read_kb":22414,"num_write":776,"num_write_kb":15855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.d","version":"176'482","reported_seq":14507,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.540594+0000","last_change":"2024-09-13T09:28:51.313612+0000","last_active":"2024-09-13T09:28:51.540594+0000","last_peered":"2024-09-13T09:28:51.540594+0000","last_clean":"2024-09-13T09:28:51.540594+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:28:51.540594+0000","last_undegraded":"2024-09-13T09:28:51.540594+0000","last_fullsized":"2024-09-13T09:28:51.540594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027639000000000001,"stat_sum":{"num_bytes":11743658,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":13905,"num_read_kb":36195,"num_write":533,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.e","version":"175'1189","reported_seq":8722,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.321432+0000","last_change":"2024-09-13T09:28:51.321720+0000","last_active":"2024-09-13T09:28:51.321432+0000","last_peered":"2024-09-13T09:28:51.321432+0000","last_clean":"2024-09-13T09:28:51.321432+0000","last_became_active":"2024-09-13T09:28:12.655468+0000","last_became_peered":"2024-09-13T09:28:12.655468+0000","last_unstale":"2024-09-13T09:28:51.321432+0000","last_undegraded":"2024-09-13T09:28:51.321432+0000","last_fullsized":"2024-09-13T09:28:51.321432+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:28.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010720564,"stat_sum":{"num_bytes":14927683,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":11018,"num_read_kb":24057,"num_write":1826,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":21465319,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.f","version":"175'468","reported_seq":6831,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.312553+0000","last_change":"2024-09-13T09:28:51.313033+0000","last_active":"2024-09-13T09:28:51.312553+0000","last_peered":"2024-09-13T09:28:51.312553+0000","last_clean":"2024-09-13T09:28:51.312553+0000","last_became_active":"2024-09-13T09:28:11.651572+0000","last_became_peered":"2024-09-13T09:28:11.651572+0000","last_unstale":"2024-09-13T09:28:51.312553+0000","last_undegraded":"2024-09-13T09:28:51.312553+0000","last_fullsized":"2024-09-13T09:28:51.312553+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:41.482908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021735830000000002,"stat_sum":{"num_bytes":23207898,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":5644,"num_read_kb":30405,"num_write":497,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":21246510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.10","version":"176'862","reported_seq":3928,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.317142+0000","last_change":"2024-09-13T09:28:51.318565+0000","last_active":"2024-09-13T09:28:51.317142+0000","last_peered":"2024-09-13T09:28:51.317142+0000","last_clean":"2024-09-13T09:28:51.317142+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:28:51.317142+0000","last_undegraded":"2024-09-13T09:28:51.317142+0000","last_fullsized":"2024-09-13T09:28:51.317142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0090375049999999995,"stat_sum":{"num_bytes":15724894,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":4961,"num_read_kb":24561,"num_write":1185,"num_write_kb":10611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.11","version":"176'869","reported_seq":8780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.329759+0000","last_change":"2024-09-13T09:28:51.330054+0000","last_active":"2024-09-13T09:28:51.329759+0000","last_peered":"2024-09-13T09:28:51.329759+0000","last_clean":"2024-09-13T09:28:51.329759+0000","last_became_active":"2024-09-13T09:28:13.679908+0000","last_became_peered":"2024-09-13T09:28:13.679908+0000","last_unstale":"2024-09-13T09:28:51.329759+0000","last_undegraded":"2024-09-13T09:28:51.329759+0000","last_fullsized":"2024-09-13T09:28:51.329759+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.428846+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018367508000000001,"stat_sum":{"num_bytes":18449923,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":10960,"num_read_kb":26455,"num_write":1020,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15533096,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.12","version":"175'547","reported_seq":5979,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.305439+0000","last_change":"2024-09-13T09:28:51.305876+0000","last_active":"2024-09-13T09:28:51.305439+0000","last_peered":"2024-09-13T09:28:51.305439+0000","last_clean":"2024-09-13T09:28:51.305439+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:28:51.305439+0000","last_undegraded":"2024-09-13T09:28:51.305439+0000","last_fullsized":"2024-09-13T09:28:51.305439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087238699999999997,"stat_sum":{"num_bytes":5376716,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":5606,"num_read_kb":26770,"num_write":660,"num_write_kb":15474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.13","version":"175'1062","reported_seq":9209,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.321756+0000","last_change":"2024-09-13T09:28:51.321961+0000","last_active":"2024-09-13T09:28:51.321756+0000","last_peered":"2024-09-13T09:28:51.321756+0000","last_clean":"2024-09-13T09:28:51.321756+0000","last_became_active":"2024-09-13T09:28:38.185466+0000","last_became_peered":"2024-09-13T09:28:38.185466+0000","last_unstale":"2024-09-13T09:28:51.321756+0000","last_undegraded":"2024-09-13T09:28:51.321756+0000","last_fullsized":"2024-09-13T09:28:51.321756+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:54.167187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010398917000000001,"stat_sum":{"num_bytes":25103583,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":11063,"num_read_kb":35055,"num_write":1348,"num_write_kb":11781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":657,"num_bytes_recovered":28570772,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":555396600,"num_objects":486,"num_object_clones":73,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":93,"num_read":247576,"num_read_kb":750637,"num_write":30729,"num_write_kb":438878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":385387783,"num_keys_recovered":1247,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22937600,"data_stored":199683503,"data_compressed":11565479,"data_compressed_allocated":15511552,"data_compressed_original":193133016,"omap_allocated":33366,"internal_metadata":0},"log_size":24336,"ondisk_log_size":24336,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50148,"kb_used_data":5080,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321692,"statfs":{"total":96636764160,"available":96585412608,"internally_reserved":0,"allocated":5201920,"data_stored":19504337,"data_compressed":932542,"data_compressed_allocated":1175552,"data_compressed_original":16580107,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.224,"5min":1.256,"15min":1.256},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.77100000000000002},"min":{"1min":0.46999999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.752,"15min":0.752},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0289999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.622},{"interface":"front","average":{"1min":0.755,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.2470000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.875,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.2649999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1910000000000001,"15min":1.2529999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.484,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.85099999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":0.96499999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.01,"15min":1.0549999999999999},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.409,"5min":1.496,"15min":1.496},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98999999999999999,"15min":1.016},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5009999999999999,"5min":1.573,"15min":1.573},"last":0.997}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39000,"kb_used_data":3788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332840,"statfs":{"total":96636764160,"available":96596828160,"internally_reserved":0,"allocated":3878912,"data_stored":11113142,"data_compressed":530446,"data_compressed_allocated":716800,"data_compressed_original":9016238,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.84399999999999997,"15min":0.88800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.3069999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.75800000000000001,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.2199999999999998,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":0.98899999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.91500000000000004,"15min":0.96099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":3.323,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.501,"5min":0.371,"15min":0.371},"max":{"1min":1.2869999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1970000000000001,"5min":5.149,"15min":5.149},"last":1.018},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2490000000000001,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.96499999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":1.083},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.874}]},{"osd":7,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91600000000000004,"15min":0.90700000000000003},"min":{"1min":0.41199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.365,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.24,"15min":1.3640000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.0579999999999998,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.83399999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67628,"kb_used_data":4504,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94304212,"statfs":{"total":96636764160,"available":96567513088,"internally_reserved":0,"allocated":4612096,"data_stored":17220338,"data_compressed":1059261,"data_compressed_allocated":1454080,"data_compressed_original":15313496,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.71499999999999997,"15min":0.74399999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.034,"5min":1.462,"15min":1.462},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.78100000000000003,"15min":0.80700000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.091,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.038,"5min":1.992,"15min":1.992},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.78000000000000003,"15min":0.81499999999999995},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1060000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.77800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.78600000000000003},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.86099999999999999},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.153,"5min":1.242,"15min":1.242},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.88500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.48},{"interface":"front","average":{"1min":0.876,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.503,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.393,"5min":1.554,"15min":1.554},"last":1.127}]},{"osd":6,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.91300000000000003,"15min":0.95899999999999996},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4079999999999999,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.93999999999999995,"15min":0.97699999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5569999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.089}]},{"osd":7,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97599999999999998,"15min":0.98999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3049999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.093},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3660000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.335}]}]},{"osd":4,"up_from":0,"seq":743029342210,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":74,"seq":317827579927,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78312,"kb_used_data":5204,"kb_used_omap":14,"kb_used_meta":73073,"kb_avail":94293528,"statfs":{"total":96636764160,"available":96556572672,"internally_reserved":0,"allocated":5328896,"data_stored":29173723,"data_compressed":1670532,"data_compressed_allocated":2199552,"data_compressed_original":27344896,"omap_allocated":14794,"internal_metadata":74827318},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.496},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.627},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.93600000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.878}]},{"osd":7,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.86399999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61524,"kb_used_data":4168,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310316,"statfs":{"total":96636764160,"available":96573763584,"internally_reserved":0,"allocated":4268032,"data_stored":14559232,"data_compressed":911418,"data_compressed_allocated":1384448,"data_compressed_original":12967367,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.97299999999999998,"5min":1.484,"15min":1.484},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.67300000000000004},"min":{"1min":0.30499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":1.335,"15min":1.335},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.97499999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.98199999999999998,"15min":1.0009999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3420000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.306,"5min":1.587,"15min":1.587},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":1.337},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.3049999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.0469999999999999,"15min":1.0920000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.169},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0049999999999999,"15min":1.038},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1990000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.2}]},{"osd":7,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0529999999999999},"min":{"1min":0.66900000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.845,"5min":2.048,"15min":2.048},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.111,"15min":1.155},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.607,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.272}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61884,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94309956,"statfs":{"total":96636764160,"available":96573394944,"internally_reserved":0,"allocated":4636672,"data_stored":18555751,"data_compressed":1179017,"data_compressed_allocated":1585152,"data_compressed_original":16801792,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":0.72299999999999998,"15min":0.66900000000000004},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.245},"max":{"1min":12.441000000000001,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.5},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.32600000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1819999999999999,"5min":1.333,"15min":1.333},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:28:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":12.555,"5min":12.555,"15min":12.555},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1950000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.052},{"interface":"front","average":{"1min":3.133,"5min":1.4350000000000001,"15min":1.1519999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":49.905999999999999,"5min":49.905999999999999,"15min":49.905999999999999},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.238},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":1.085}]},{"osd":6,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.139,"15min":1.111},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":12.519,"5min":12.519,"15min":12.519},"last":1.196},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0469999999999999,"15min":1.0760000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3200000000000001,"5min":1.514,"15min":1.514},"last":1.123}]},{"osd":7,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.194,"15min":1.147},"min":{"1min":0.60799999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":14.143000000000001,"5min":14.143000000000001,"15min":14.143000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.087,"15min":1.1220000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.0129999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56888,"kb_used_data":3624,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94314952,"statfs":{"total":96636764160,"available":96578510848,"internally_reserved":0,"allocated":3710976,"data_stored":10153682,"data_compressed":546587,"data_compressed_allocated":827392,"data_compressed_original":8495104,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.61799999999999999},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.37},{"interface":"front","average":{"1min":1.163,"5min":0.78900000000000003,"15min":0.72699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":8.5869999999999997,"5min":8.5869999999999997,"15min":8.5869999999999997},"last":0.623}]},{"osd":1,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.502},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.73999999999999999,"15min":0.69399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.4820000000000002,"5min":4.4820000000000002,"15min":4.4820000000000002},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.292,"5min":1.996,"15min":1.996},"last":1.02},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6379999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.131},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98399999999999999,"15min":1.006},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3720000000000001,"5min":1.651,"15min":1.651},"last":1.081}]},{"osd":6,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.0609999999999999,"15min":1.0069999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.8490000000000002,"5min":8.8490000000000002,"15min":8.8490000000000002},"last":1.181},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96399999999999997,"15min":0.97299999999999998},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.149,"15min":1.1040000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":10.659000000000001,"5min":10.659000000000001,"15min":10.659000000000001},"last":1.099},{"interface":"front","average":{"1min":1.202,"5min":1.04,"15min":1.0149999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.9369999999999998,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.052}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2142208,"data_stored":15376441,"data_compressed":934030,"data_compressed_allocated":1286144,"data_compressed_original":14704640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2088960,"data_stored":12952283,"data_compressed":878411,"data_compressed_allocated":1339392,"data_compressed_original":12393927,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1609728,"data_stored":9143365,"data_compressed":546587,"data_compressed_allocated":827392,"data_compressed_original":8495104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":23778307,"data_compressed":1352205,"data_compressed_allocated":1863680,"data_compressed_original":23003136,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7294976,"data_stored":94550550,"data_compressed":5365996,"data_compressed_allocated":6901760,"data_compressed_original":94220288,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2404352,"data_stored":16115050,"data_compressed":1059261,"data_compressed_allocated":1454080,"data_compressed_original":15313496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1765376,"data_stored":10096561,"data_compressed":530446,"data_compressed_allocated":716800,"data_compressed_original":9016238,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2789376,"data_stored":17670946,"data_compressed":898543,"data_compressed_allocated":1122304,"data_compressed_original":15986187,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:28:54.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:28:54.764 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:54.764 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:54.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:54.764 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:97976 3.c 3:34937af9:::rbd_data.111838d2a860.0000000000000004:17 [read 367616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.678457881, 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050546960000000002, 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 97976 2024-09-13T09:28:54.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082087+0000", 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082090+0000", 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.464e-06 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082094+0000", 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8770000000000001e-06 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.082101+0000", 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2589999999999997e-06 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129017+0000", 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.046916272000000002 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.129103+0000", 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5649000000000006e-05 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:52.132634+0000", 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035317679999999998 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.769 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:125639 3.12 3:4ee51019:::rbd_data.1118d0eacce0.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 395925~40124] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 103.45521728999999, 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49478704099999998, 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 125639 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305328+0000", 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305326+0000", 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305375+0000", 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8612999999999997e-05 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305399+0000", 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4476000000000001e-05 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305411+0000", 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1739000000000001e-05 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305516+0000", 2024-09-13T09:28:54.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105128 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.305536+0000", 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0072999999999999e-05 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.797758+0000", 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.49222170500000001 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798030+0000", 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027177800000000003 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.798386+0000", 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035644600000000001 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.799781+0000", 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001394798 2024-09-13T09:28:54.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800007+0000", 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022573999999999999 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800060+0000", 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3779000000000001e-05 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:11.800115+0000", 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4583000000000002e-05 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:126493 3.12 3:4e4abf68:::rbd_data.1118bda513e1.0000000000000002:head [sparse-read 0~585819] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 99.928004083999994, 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2358026099999999, 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 126493 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832541+0000", 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832539+0000", 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832551+0000", 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2244e-05 2024-09-13T09:28:54.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832593+0000", 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1746999999999999e-05 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832632+0000", 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8992000000000002e-05 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832785+0000", 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000153382 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.832847+0000", 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1398999999999999e-05 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835922+0000", 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030752779999999999 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:14.835935+0000", 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2901000000000001e-05 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066500+0000", 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2305656540000001 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.066774+0000", 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027380200000000002 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:16.068344+0000", 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001569626 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168638 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10715136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.783 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.668757135, 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049862933999999998, 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168638 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091788+0000", 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091790+0000", 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3360000000000002e-06 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091795+0000", 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8010000000000003e-06 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091799+0000", 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1459999999999998e-06 2024-09-13T09:28:54.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141530+0000", 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049731113 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141593+0000", 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3074e-05 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141651+0000", 2024-09-13T09:28:54.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7976000000000002e-05 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168639 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.668740579999998, 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050003361000000003, 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168639 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091805+0000", 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091802+0000", 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091807+0000", 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091812+0000", 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.502e-06 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091816+0000", 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141684+0000", 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049867828000000003 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141751+0000", 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6830999999999998e-05 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141808+0000", 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7250000000000002e-05 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168640 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10756096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.668725516000002, 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142574000000002, 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168640 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091820+0000", 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091819+0000", 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091822+0000", 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1339999999999998e-06 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091826+0000", 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3640000000000001e-06 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.091830+0000", 2024-09-13T09:28:54.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0720000000000001e-06 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141844+0000", 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050013811999999998 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141906+0000", 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1865999999999993e-05 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.141962+0000", 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6752000000000001e-05 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168643 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10817536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.667993752999998, 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049885106999999998, 2024-09-13T09:28:54.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168643 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092552+0000", 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092551+0000", 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092555+0000", 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9360000000000001e-06 2024-09-13T09:28:54.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092567+0000", 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2342e-05 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092572+0000", 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142317+0000", 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049745523999999999 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142379+0000", 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1950000000000001e-05 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142437+0000", 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7707000000000002e-05 2024-09-13T09:28:54.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168644 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10838016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.667969847000002, 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050021878999999998, 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168644 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092575+0000", 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092577+0000", 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092582+0000", 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7820000000000001e-06 2024-09-13T09:28:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.092589+0000", 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9489999999999999e-06 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142469+0000", 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049879824000000003 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142534+0000", 2024-09-13T09:28:54.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999999e-05 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142597+0000", 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2940000000000004e-05 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168647 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10899456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:54.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.667277847000001, 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049852212999999999, 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168647 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093267+0000", 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093270+0000", 2024-09-13T09:28:54.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4660000000000002e-06 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093274+0000", 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6299999999999997e-06 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093278+0000", 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8720000000000004e-06 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.142988+0000", 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049710441000000001 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143066+0000", 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8020000000000002e-05 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143120+0000", 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168648 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10919936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.667263810999998, 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050014306000000001, 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168648 2024-09-13T09:28:54.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093282+0000", 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093281+0000", 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093283+0000", 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2859999999999998e-06 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093288+0000", 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093292+0000", 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.788e-06 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143170+0000", 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049878179000000002 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143238+0000", 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8553999999999994e-05 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143296+0000", 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7491000000000002e-05 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168649 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10940416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.667145433999998, 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050062137999999999, 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168649 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093400+0000", 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093295+0000", 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998951 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093403+0000", 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108105 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093409+0000", 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3659999999999997e-06 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093414+0000", 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6700000000000002e-06 2024-09-13T09:28:54.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143335+0000", 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049920496000000002 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143400+0000", 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4993999999999995e-05 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143462+0000", 2024-09-13T09:28:54.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2441999999999996e-05 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168650 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 10960896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.666952301999999, 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050102019999999997, 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168650 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093593+0000", 2024-09-13T09:28:54.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093418+0000", 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999825 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093596+0000", 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017842199999999999 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093604+0000", 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.093609+0000", 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.87e-06 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143512+0000", 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049903852999999998 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143615+0000", 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102593 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.143695+0000", 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0019999999999996e-05 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168666 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11268096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.663017181999997, 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049785755000000001, 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168666 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097528+0000", 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097529+0000", 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097532+0000", 2024-09-13T09:28:54.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7609999999999999e-06 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097535+0000", 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.48e-06 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147168+0000", 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049633799999999999 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147244+0000", 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5352000000000006e-05 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147314+0000", 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0166000000000005e-05 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168667 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11288576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.663008496000003, 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049944720999999997, 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168667 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097537+0000", 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097536+0000", 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097538+0000", 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4160000000000001e-06 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097541+0000", 2024-09-13T09:28:54.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4979999999999998e-06 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097544+0000", 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.424e-06 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147351+0000", 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049806823 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147426+0000", 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5741000000000005e-05 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147482+0000", 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5211999999999999e-05 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168668 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11309056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.662999136000003, 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050142884999999998, 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168668 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097546+0000", 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:54.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097547+0000", 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.297e-06 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097550+0000", 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8770000000000002e-06 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097553+0000", 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.559e-06 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147519+0000", 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.049966658999999997 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147611+0000", 2024-09-13T09:28:54.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1397000000000006e-05 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147689+0000", 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8324000000000004e-05 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.839 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168669 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11329536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.662990356999998, 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050306801999999998, 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168669 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097555+0000", 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097556+0000", 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3230000000000001e-06 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097559+0000", 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097561+0000", 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4870000000000001e-06 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147728+0000", 2024-09-13T09:28:54.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050166428999999998 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147783+0000", 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147862+0000", 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8984000000000001e-05 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168670 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11350016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.662981778000002, 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050450338999999997, 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168670 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097564+0000", 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097563+0000", 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097568+0000", 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7260000000000002e-06 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097571+0000", 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:28:54.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097574+0000", 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5620000000000002e-06 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147905+0000", 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050331168000000003 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.147971+0000", 2024-09-13T09:28:54.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6011000000000002e-05 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148014+0000", 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2824000000000002e-05 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168671 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11370496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:54.849 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.662969093999997, 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050591117999999997, 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168671 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097576+0000", 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:54.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097577+0000", 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5260000000000001e-06 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097580+0000", 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7650000000000002e-06 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097583+0000", 2024-09-13T09:28:54.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6429999999999999e-06 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148039+0000", 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050455834999999997 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148095+0000", 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5803000000000001e-05 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148167+0000", 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2809e-05 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:168672 3.5 3:a0bc4324:::rbd_data.111827e6de71.0000000000000007:30 [read 11390976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 48.662960202000001, 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050685493999999998, 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 168672 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097585+0000", 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097586+0000", 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.364e-06 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097589+0000", 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7489999999999999e-06 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.097592+0000", 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9220000000000001e-06 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148187+0000", 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.050594627000000003 2024-09-13T09:28:54.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148230+0000", 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2964999999999999e-05 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.148271+0000", 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1094999999999999e-05 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:182783 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e136)", 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097133325999998, 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99356371799999998, 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 182783 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.859 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663412+0000", 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663410+0000", 2024-09-13T09:28:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663420+0000", 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0097e-05 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663440+0000", 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9559999999999999e-05 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663448+0000", 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.824e-06 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663515+0000", 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6964999999999993e-05 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:11.663527+0000", 2024-09-13T09:28:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2295e-05 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656700+0000", 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99317316300000003 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656806+0000", 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105889 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:12.656976+0000", 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016978699999999999 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:54.864 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:54.866 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:54.866 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:54.866 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:54.866 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:54.866 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:54.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:54.867 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:54.861+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:54.873 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5: {} 2024-09-13T09:28:54.896 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:54.918 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:54.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:54.966 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:28:54.961+0000 7ff52738e700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 22314) UID: 0 2024-09-13T09:28:55.023 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.023 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:55.061 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:28:55.064 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:28:55.077 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-13T09:28:54.569601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555986987,"num_objects":490,"num_object_clones":73,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":93,"num_read":247692,"num_read_kb":750737,"num_write":30890,"num_write_kb":441296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6793,"num_bytes_recovered":385387821,"num_keys_recovered":1247,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24456,"ondisk_log_size":24456,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":415384,"kb_used_data":30896,"kb_used_omap":23,"kb_used_meta":384360,"kb_avail":660187496,"statfs":{"total":676457349120,"available":676031995904,"internally_reserved":0,"allocated":31637504,"data_stored":120280205,"data_compressed":6829803,"data_compressed_allocated":9342976,"data_compressed_original":106519000,"omap_allocated":24352,"internal_metadata":393584864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-188557283,"num_objects":-191,"num_object_clones":-40,"num_object_copies":-382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-191,"num_whiteouts":-23,"num_read":1279,"num_read_kb":1097,"num_write":305,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001181"},"pg_stats":[{"pgid":"3.1f","version":"176'517","reported_seq":2078,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.314771+0000","last_change":"2024-09-13T09:28:51.315176+0000","last_active":"2024-09-13T09:28:51.314771+0000","last_peered":"2024-09-13T09:28:51.314771+0000","last_clean":"2024-09-13T09:28:51.314771+0000","last_became_active":"2024-09-13T09:26:39.539274+0000","last_became_peered":"2024-09-13T09:26:39.539274+0000","last_unstale":"2024-09-13T09:28:51.314771+0000","last_undegraded":"2024-09-13T09:28:51.314771+0000","last_fullsized":"2024-09-13T09:28:51.314771+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:37:51.182303+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0097452289999999994,"stat_sum":{"num_bytes":12223234,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3176,"num_read_kb":16241,"num_write":511,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1e","version":"175'813","reported_seq":5689,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.397717+0000","last_change":"2024-09-13T09:28:50.202702+0000","last_active":"2024-09-13T09:28:50.397717+0000","last_peered":"2024-09-13T09:28:50.397717+0000","last_clean":"2024-09-13T09:28:50.397717+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:28:50.397717+0000","last_undegraded":"2024-09-13T09:28:50.397717+0000","last_fullsized":"2024-09-13T09:28:50.397717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01203196,"stat_sum":{"num_bytes":8497000,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":6798,"num_read_kb":14454,"num_write":1116,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.1d","version":"176'618","reported_seq":5377,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.326769+0000","last_change":"2024-09-13T09:28:51.327075+0000","last_active":"2024-09-13T09:28:51.326769+0000","last_peered":"2024-09-13T09:28:51.326769+0000","last_clean":"2024-09-13T09:28:51.326769+0000","last_became_active":"2024-09-13T09:28:12.655560+0000","last_became_peered":"2024-09-13T09:28:12.655560+0000","last_unstale":"2024-09-13T09:28:51.326769+0000","last_undegraded":"2024-09-13T09:28:51.326769+0000","last_fullsized":"2024-09-13T09:28:51.326769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:17.740965+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010390780000000001,"stat_sum":{"num_bytes":19666007,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":5233,"num_read_kb":24957,"num_write":713,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":21645529,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1c","version":"176'772","reported_seq":6070,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.385882+0000","last_change":"2024-09-13T09:28:51.317727+0000","last_active":"2024-09-13T09:28:51.385882+0000","last_peered":"2024-09-13T09:28:51.385882+0000","last_clean":"2024-09-13T09:28:51.385882+0000","last_became_active":"2024-09-13T09:28:12.654993+0000","last_became_peered":"2024-09-13T09:28:12.654993+0000","last_unstale":"2024-09-13T09:28:51.385882+0000","last_undegraded":"2024-09-13T09:28:51.385882+0000","last_fullsized":"2024-09-13T09:28:51.385882+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:26.519049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001236661,"stat_sum":{"num_bytes":17355159,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":6207,"num_read_kb":19081,"num_write":1075,"num_write_kb":14319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":24632032,"num_keys_recovered":228,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1b","version":"175'673","reported_seq":6763,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.307534+0000","last_change":"2024-09-13T09:28:51.309071+0000","last_active":"2024-09-13T09:28:51.307534+0000","last_peered":"2024-09-13T09:28:51.307534+0000","last_clean":"2024-09-13T09:28:51.307534+0000","last_became_active":"2024-09-13T09:28:11.652761+0000","last_became_peered":"2024-09-13T09:28:11.652761+0000","last_unstale":"2024-09-13T09:28:51.307534+0000","last_undegraded":"2024-09-13T09:28:51.307534+0000","last_fullsized":"2024-09-13T09:28:51.307534+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:13:09.566533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039085379999999996,"stat_sum":{"num_bytes":3750401,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":8005,"num_read_kb":16391,"num_write":692,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1a","version":"172'619","reported_seq":13078,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.511347+0000","last_change":"2024-09-13T09:28:51.317609+0000","last_active":"2024-09-13T09:28:51.511347+0000","last_peered":"2024-09-13T09:28:51.511347+0000","last_clean":"2024-09-13T09:28:51.511347+0000","last_became_active":"2024-09-13T09:28:38.183181+0000","last_became_peered":"2024-09-13T09:28:38.183181+0000","last_unstale":"2024-09-13T09:28:51.511347+0000","last_undegraded":"2024-09-13T09:28:51.511347+0000","last_fullsized":"2024-09-13T09:28:51.511347+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:13:08.070603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013251249999999999,"stat_sum":{"num_bytes":34090509,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":13366,"num_read_kb":23715,"num_write":697,"num_write_kb":9206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":583,"num_bytes_recovered":22514520,"num_keys_recovered":115,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.19","version":"175'633","reported_seq":3650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.316215+0000","last_change":"2024-09-13T09:28:51.316651+0000","last_active":"2024-09-13T09:28:51.316215+0000","last_peered":"2024-09-13T09:28:51.316215+0000","last_clean":"2024-09-13T09:28:51.316215+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:28:51.316215+0000","last_undegraded":"2024-09-13T09:28:51.316215+0000","last_fullsized":"2024-09-13T09:28:51.316215+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055758900000000004,"stat_sum":{"num_bytes":20467685,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":4338,"num_read_kb":14826,"num_write":615,"num_write_kb":8147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.18","version":"175'634","reported_seq":5517,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.185653+0000","last_change":"2024-09-13T09:28:50.185653+0000","last_active":"2024-09-13T09:28:50.185653+0000","last_peered":"2024-09-13T09:28:50.185653+0000","last_clean":"2024-09-13T09:28:50.185653+0000","last_became_active":"2024-09-13T09:28:37.183228+0000","last_became_peered":"2024-09-13T09:28:37.183228+0000","last_unstale":"2024-09-13T09:28:50.185653+0000","last_undegraded":"2024-09-13T09:28:50.185653+0000","last_fullsized":"2024-09-13T09:28:50.185653+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:34.997664+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0081704910000000002,"stat_sum":{"num_bytes":7771224,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":7146,"num_read_kb":22085,"num_write":609,"num_write_kb":15608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.17","version":"172'760","reported_seq":2569,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.310590+0000","last_change":"2024-09-13T09:28:51.310947+0000","last_active":"2024-09-13T09:28:51.310590+0000","last_peered":"2024-09-13T09:28:51.310590+0000","last_clean":"2024-09-13T09:28:51.310590+0000","last_became_active":"2024-09-13T09:26:42.574484+0000","last_became_peered":"2024-09-13T09:26:42.574484+0000","last_unstale":"2024-09-13T09:28:51.310590+0000","last_undegraded":"2024-09-13T09:28:51.310590+0000","last_fullsized":"2024-09-13T09:28:51.310590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":760,"log_dups_size":0,"ondisk_log_size":760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:48:47.404587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0053277510000000004,"stat_sum":{"num_bytes":2255815,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":4228,"num_read_kb":9889,"num_write":846,"num_write_kb":4773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.16","version":"173'581","reported_seq":6059,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.317881+0000","last_change":"2024-09-13T09:28:51.318342+0000","last_active":"2024-09-13T09:28:51.317881+0000","last_peered":"2024-09-13T09:28:51.317881+0000","last_clean":"2024-09-13T09:28:51.317881+0000","last_became_active":"2024-09-13T09:25:53.176738+0000","last_became_peered":"2024-09-13T09:25:53.176738+0000","last_unstale":"2024-09-13T09:28:51.317881+0000","last_undegraded":"2024-09-13T09:28:51.317881+0000","last_fullsized":"2024-09-13T09:28:51.317881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:22.020526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015313200000000001,"stat_sum":{"num_bytes":24461347,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":5787,"num_read_kb":36038,"num_write":669,"num_write_kb":20370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.15","version":"176'486","reported_seq":10460,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.489901+0000","last_change":"2024-09-13T09:28:51.321372+0000","last_active":"2024-09-13T09:28:51.489901+0000","last_peered":"2024-09-13T09:28:51.489901+0000","last_clean":"2024-09-13T09:28:51.489901+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:28:51.489901+0000","last_undegraded":"2024-09-13T09:28:51.489901+0000","last_fullsized":"2024-09-13T09:28:51.489901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010612369,"stat_sum":{"num_bytes":7628313,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":10262,"num_read_kb":33263,"num_write":488,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.14","version":"176'599","reported_seq":6388,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.326498+0000","last_change":"2024-09-13T09:28:51.326733+0000","last_active":"2024-09-13T09:28:51.326498+0000","last_peered":"2024-09-13T09:28:51.326498+0000","last_clean":"2024-09-13T09:28:51.326498+0000","last_became_active":"2024-09-13T09:28:11.648374+0000","last_became_peered":"2024-09-13T09:28:11.648374+0000","last_unstale":"2024-09-13T09:28:51.326498+0000","last_undegraded":"2024-09-13T09:28:51.326498+0000","last_fullsized":"2024-09-13T09:28:51.326498+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:26:26.377449+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015744873,"stat_sum":{"num_bytes":23863900,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":6502,"num_read_kb":13665,"num_write":644,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":8018413,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.7","version":"0'0","reported_seq":186,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.177564+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:28:50.177564+0000","last_peered":"2024-09-13T09:28:50.177564+0000","last_clean":"2024-09-13T09:28:50.177564+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:28:50.177564+0000","last_undegraded":"2024-09-13T09:28:50.177564+0000","last_fullsized":"2024-09-13T09:28:50.177564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"175'520","reported_seq":2788,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.318632+0000","last_change":"2024-09-13T09:28:51.319044+0000","last_active":"2024-09-13T09:28:51.318632+0000","last_peered":"2024-09-13T09:28:51.318632+0000","last_clean":"2024-09-13T09:28:51.318632+0000","last_became_active":"2024-09-13T09:27:05.753131+0000","last_became_peered":"2024-09-13T09:27:05.753131+0000","last_unstale":"2024-09-13T09:28:51.318632+0000","last_undegraded":"2024-09-13T09:28:51.318632+0000","last_fullsized":"2024-09-13T09:28:51.318632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:13:21.776084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021241599999999999,"stat_sum":{"num_bytes":32070734,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":2305,"num_read_kb":6499,"num_write":566,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.6","version":"0'0","reported_seq":194,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.177843+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:28:50.177843+0000","last_peered":"2024-09-13T09:28:50.177843+0000","last_clean":"2024-09-13T09:28:50.177843+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:28:50.177843+0000","last_undegraded":"2024-09-13T09:28:50.177843+0000","last_fullsized":"2024-09-13T09:28:50.177843+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"175'902","reported_seq":4421,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.316966+0000","last_change":"2024-09-13T09:28:51.317252+0000","last_active":"2024-09-13T09:28:51.316966+0000","last_peered":"2024-09-13T09:28:51.316966+0000","last_clean":"2024-09-13T09:28:51.316966+0000","last_became_active":"2024-09-13T09:28:39.186243+0000","last_became_peered":"2024-09-13T09:28:39.186243+0000","last_unstale":"2024-09-13T09:28:51.316966+0000","last_undegraded":"2024-09-13T09:28:51.316966+0000","last_fullsized":"2024-09-13T09:28:51.316966+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":902,"log_dups_size":0,"ondisk_log_size":902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:31:40.874996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001060269,"stat_sum":{"num_bytes":5584529,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":5847,"num_read_kb":19431,"num_write":1182,"num_write_kb":22716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-13T09:28:49.297227+0000","last_change":"2024-09-13T09:28:33.240144+0000","last_active":"2024-09-13T09:28:49.297227+0000","last_peered":"2024-09-13T09:28:49.297227+0000","last_clean":"2024-09-13T09:28:49.297227+0000","last_became_active":"2024-09-13T09:28:33.239677+0000","last_became_peered":"2024-09-13T09:28:33.239677+0000","last_unstale":"2024-09-13T09:28:49.297227+0000","last_undegraded":"2024-09-13T09:28:49.297227+0000","last_fullsized":"2024-09-13T09:28:49.297227+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:04:26.140186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"175'867","reported_seq":9056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.201921+0000","last_change":"2024-09-13T09:28:50.202321+0000","last_active":"2024-09-13T09:28:50.201921+0000","last_peered":"2024-09-13T09:28:50.201921+0000","last_clean":"2024-09-13T09:28:50.201921+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:28:50.201921+0000","last_undegraded":"2024-09-13T09:28:50.201921+0000","last_fullsized":"2024-09-13T09:28:50.201921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011828231999999999,"stat_sum":{"num_bytes":23403538,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1122,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"1.4","version":"0'0","reported_seq":130,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-13T09:28:49.296161+0000","last_change":"2024-09-13T09:27:02.707694+0000","last_active":"2024-09-13T09:28:49.296161+0000","last_peered":"2024-09-13T09:28:49.296161+0000","last_clean":"2024-09-13T09:28:49.296161+0000","last_became_active":"2024-09-13T09:27:02.707392+0000","last_became_peered":"2024-09-13T09:27:02.707392+0000","last_unstale":"2024-09-13T09:28:49.296161+0000","last_undegraded":"2024-09-13T09:28:49.296161+0000","last_fullsized":"2024-09-13T09:28:49.296161+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:24.366372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'845","reported_seq":5884,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.498176+0000","last_change":"2024-09-13T09:28:51.309932+0000","last_active":"2024-09-13T09:28:51.498176+0000","last_peered":"2024-09-13T09:28:51.498176+0000","last_clean":"2024-09-13T09:28:51.498176+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:28:51.498176+0000","last_undegraded":"2024-09-13T09:28:51.498176+0000","last_fullsized":"2024-09-13T09:28:51.498176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046189259999999998,"stat_sum":{"num_bytes":5288604,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":7912,"num_read_kb":24806,"num_write":998,"num_write_kb":10476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.1","version":"176'1366","reported_seq":3870,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.543257+0000","last_change":"2024-09-13T09:28:51.306517+0000","last_active":"2024-09-13T09:28:51.543257+0000","last_peered":"2024-09-13T09:28:51.543257+0000","last_clean":"2024-09-13T09:28:51.543257+0000","last_became_active":"2024-09-13T09:28:12.650128+0000","last_became_peered":"2024-09-13T09:28:12.650128+0000","last_unstale":"2024-09-13T09:28:51.543257+0000","last_undegraded":"2024-09-13T09:28:51.543257+0000","last_fullsized":"2024-09-13T09:28:51.543257+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1366,"log_dups_size":0,"ondisk_log_size":1366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:23.557202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001514903,"stat_sum":{"num_bytes":1200226,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":7691,"num_read_kb":20013,"num_write":2047,"num_write_kb":7019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":10575408,"num_keys_recovered":140,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"2.0","version":"19'117","reported_seq":316,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.188706+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:28:50.188706+0000","last_peered":"2024-09-13T09:28:50.188706+0000","last_clean":"2024-09-13T09:28:50.188706+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:28:50.188706+0000","last_undegraded":"2024-09-13T09:28:50.188706+0000","last_fullsized":"2024-09-13T09:28:50.188706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":174,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.190791+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:28:50.190791+0000","last_peered":"2024-09-13T09:28:50.190791+0000","last_clean":"2024-09-13T09:28:50.190791+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:28:50.190791+0000","last_undegraded":"2024-09-13T09:28:50.190791+0000","last_fullsized":"2024-09-13T09:28:50.190791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"175'985","reported_seq":9287,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.317405+0000","last_change":"2024-09-13T09:28:51.317679+0000","last_active":"2024-09-13T09:28:51.317405+0000","last_peered":"2024-09-13T09:28:51.317405+0000","last_clean":"2024-09-13T09:28:51.317405+0000","last_became_active":"2024-09-13T09:28:39.186937+0000","last_became_peered":"2024-09-13T09:28:39.186937+0000","last_unstale":"2024-09-13T09:28:51.317405+0000","last_undegraded":"2024-09-13T09:28:51.317405+0000","last_fullsized":"2024-09-13T09:28:51.317405+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:03:51.885937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095999300000000002,"stat_sum":{"num_bytes":23323589,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":2,"num_read":10796,"num_read_kb":23352,"num_write":1403,"num_write_kb":18964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.2","version":"19'2","reported_seq":198,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.188973+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:28:50.188973+0000","last_peered":"2024-09-13T09:28:50.188973+0000","last_clean":"2024-09-13T09:28:50.188973+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:28:50.188973+0000","last_undegraded":"2024-09-13T09:28:50.188973+0000","last_fullsized":"2024-09-13T09:28:50.188973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'765","reported_seq":11108,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.500712+0000","last_change":"2024-09-13T09:28:51.310518+0000","last_active":"2024-09-13T09:28:51.500712+0000","last_peered":"2024-09-13T09:28:51.500712+0000","last_clean":"2024-09-13T09:28:51.500712+0000","last_became_active":"2024-09-13T09:28:12.654541+0000","last_became_peered":"2024-09-13T09:28:12.654541+0000","last_unstale":"2024-09-13T09:28:51.500712+0000","last_undegraded":"2024-09-13T09:28:51.500712+0000","last_fullsized":"2024-09-13T09:28:51.500712+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:51:32.252069+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0056077749999999997,"stat_sum":{"num_bytes":38321324,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":10021,"num_read_kb":38964,"num_write":855,"num_write_kb":31778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":52739384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"1.1","version":"0'0","reported_seq":190,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.189048+0000","last_change":"2024-09-13T09:26:39.538886+0000","last_active":"2024-09-13T09:28:50.189048+0000","last_peered":"2024-09-13T09:28:50.189048+0000","last_clean":"2024-09-13T09:28:50.189048+0000","last_became_active":"2024-09-13T09:26:39.538595+0000","last_became_peered":"2024-09-13T09:26:39.538595+0000","last_unstale":"2024-09-13T09:28:50.189048+0000","last_undegraded":"2024-09-13T09:28:50.189048+0000","last_fullsized":"2024-09-13T09:28:50.189048+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:26:45.501171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"175'2093","reported_seq":6932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.366692+0000","last_change":"2024-09-13T09:28:50.206159+0000","last_active":"2024-09-13T09:28:50.366692+0000","last_peered":"2024-09-13T09:28:50.366692+0000","last_clean":"2024-09-13T09:28:50.366692+0000","last_became_active":"2024-09-13T09:27:05.744803+0000","last_became_peered":"2024-09-13T09:27:05.744803+0000","last_unstale":"2024-09-13T09:28:50.366692+0000","last_undegraded":"2024-09-13T09:28:50.366692+0000","last_fullsized":"2024-09-13T09:28:50.366692+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2093,"log_dups_size":0,"ondisk_log_size":2093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:40.185419+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015829551000000001,"stat_sum":{"num_bytes":6636818,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":14692,"num_read_kb":17088,"num_write":3712,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"1.0","version":"0'0","reported_seq":188,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.188801+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:28:50.188801+0000","last_peered":"2024-09-13T09:28:50.188801+0000","last_clean":"2024-09-13T09:28:50.188801+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:28:50.188801+0000","last_undegraded":"2024-09-13T09:28:50.188801+0000","last_fullsized":"2024-09-13T09:28:50.188801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"172'625","reported_seq":3350,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.191019+0000","last_change":"2024-09-13T09:28:50.191589+0000","last_active":"2024-09-13T09:28:50.191019+0000","last_peered":"2024-09-13T09:28:50.191019+0000","last_clean":"2024-09-13T09:28:50.191019+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:28:50.191019+0000","last_undegraded":"2024-09-13T09:28:50.191019+0000","last_fullsized":"2024-09-13T09:28:50.191019+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":625,"log_dups_size":0,"ondisk_log_size":625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011978959999999999,"stat_sum":{"num_bytes":8521299,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":4377,"num_read_kb":15143,"num_write":647,"num_write_kb":7476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.9","version":"172'464","reported_seq":11900,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.177751+0000","last_change":"2024-09-13T09:28:50.178009+0000","last_active":"2024-09-13T09:28:50.177751+0000","last_peered":"2024-09-13T09:28:50.177751+0000","last_clean":"2024-09-13T09:28:50.177751+0000","last_became_active":"2024-09-13T09:28:36.178628+0000","last_became_peered":"2024-09-13T09:28:36.178628+0000","last_unstale":"2024-09-13T09:28:50.177751+0000","last_undegraded":"2024-09-13T09:28:50.177751+0000","last_fullsized":"2024-09-13T09:28:50.177751+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:26:16.388720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052350499999999996,"stat_sum":{"num_bytes":30532891,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":10498,"num_read_kb":27484,"num_write":476,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":597,"num_bytes_recovered":25814267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.a","version":"176'557","reported_seq":7214,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.505597+0000","last_change":"2024-09-13T09:28:51.321343+0000","last_active":"2024-09-13T09:28:51.505597+0000","last_peered":"2024-09-13T09:28:51.505597+0000","last_clean":"2024-09-13T09:28:51.505597+0000","last_became_active":"2024-09-13T09:28:11.646271+0000","last_became_peered":"2024-09-13T09:28:11.646271+0000","last_unstale":"2024-09-13T09:28:51.505597+0000","last_undegraded":"2024-09-13T09:28:51.505597+0000","last_fullsized":"2024-09-13T09:28:51.505597+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:41:49.883994+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011730127999999999,"stat_sum":{"num_bytes":5582864,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":6480,"num_read_kb":26569,"num_write":533,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":267,"num_bytes_recovered":13216032,"num_keys_recovered":88,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.b","version":"174'592","reported_seq":4353,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-13T09:28:50.202766+0000","last_change":"2024-09-13T09:28:50.203011+0000","last_active":"2024-09-13T09:28:50.202766+0000","last_peered":"2024-09-13T09:28:50.202766+0000","last_clean":"2024-09-13T09:28:50.202766+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:28:50.202766+0000","last_undegraded":"2024-09-13T09:28:50.202766+0000","last_fullsized":"2024-09-13T09:28:50.202766+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010154699999999999,"stat_sum":{"num_bytes":57229243,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":3,"num_read":3847,"num_read_kb":28423,"num_write":668,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"13","length":"5"}]},{"pgid":"3.c","version":"175'571","reported_seq":7529,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.316112+0000","last_change":"2024-09-13T09:28:51.316907+0000","last_active":"2024-09-13T09:28:51.316112+0000","last_peered":"2024-09-13T09:28:51.316112+0000","last_clean":"2024-09-13T09:28:51.316112+0000","last_became_active":"2024-09-13T09:27:05.745595+0000","last_became_peered":"2024-09-13T09:27:05.745595+0000","last_unstale":"2024-09-13T09:28:51.316112+0000","last_undegraded":"2024-09-13T09:28:51.316112+0000","last_fullsized":"2024-09-13T09:28:51.316112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:38.773883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00101932,"stat_sum":{"num_bytes":21135992,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":7720,"num_read_kb":22414,"num_write":776,"num_write_kb":15855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.d","version":"176'482","reported_seq":14507,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.540594+0000","last_change":"2024-09-13T09:28:51.313612+0000","last_active":"2024-09-13T09:28:51.540594+0000","last_peered":"2024-09-13T09:28:51.540594+0000","last_clean":"2024-09-13T09:28:51.540594+0000","last_became_active":"2024-09-13T09:25:53.180307+0000","last_became_peered":"2024-09-13T09:25:53.180307+0000","last_unstale":"2024-09-13T09:28:51.540594+0000","last_undegraded":"2024-09-13T09:28:51.540594+0000","last_fullsized":"2024-09-13T09:28:51.540594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:40.777816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027639000000000001,"stat_sum":{"num_bytes":11743658,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":13905,"num_read_kb":36195,"num_write":533,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.e","version":"175'1189","reported_seq":8722,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.321432+0000","last_change":"2024-09-13T09:28:51.321720+0000","last_active":"2024-09-13T09:28:51.321432+0000","last_peered":"2024-09-13T09:28:51.321432+0000","last_clean":"2024-09-13T09:28:51.321432+0000","last_became_active":"2024-09-13T09:28:12.655468+0000","last_became_peered":"2024-09-13T09:28:12.655468+0000","last_unstale":"2024-09-13T09:28:51.321432+0000","last_undegraded":"2024-09-13T09:28:51.321432+0000","last_fullsized":"2024-09-13T09:28:51.321432+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:28.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010720564,"stat_sum":{"num_bytes":14927683,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":11018,"num_read_kb":24057,"num_write":1826,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":21465319,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.f","version":"175'468","reported_seq":6831,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.312553+0000","last_change":"2024-09-13T09:28:51.313033+0000","last_active":"2024-09-13T09:28:51.312553+0000","last_peered":"2024-09-13T09:28:51.312553+0000","last_clean":"2024-09-13T09:28:51.312553+0000","last_became_active":"2024-09-13T09:28:11.651572+0000","last_became_peered":"2024-09-13T09:28:11.651572+0000","last_unstale":"2024-09-13T09:28:51.312553+0000","last_undegraded":"2024-09-13T09:28:51.312553+0000","last_fullsized":"2024-09-13T09:28:51.312553+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:24:41.482908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021735830000000002,"stat_sum":{"num_bytes":23207898,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":5644,"num_read_kb":30405,"num_write":497,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":21246510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.10","version":"176'862","reported_seq":3928,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.317142+0000","last_change":"2024-09-13T09:28:51.318565+0000","last_active":"2024-09-13T09:28:51.317142+0000","last_peered":"2024-09-13T09:28:51.317142+0000","last_clean":"2024-09-13T09:28:51.317142+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:28:51.317142+0000","last_undegraded":"2024-09-13T09:28:51.317142+0000","last_fullsized":"2024-09-13T09:28:51.317142+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0090375049999999995,"stat_sum":{"num_bytes":15724894,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":4961,"num_read_kb":24561,"num_write":1185,"num_write_kb":10611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.11","version":"176'869","reported_seq":8780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.329759+0000","last_change":"2024-09-13T09:28:51.330054+0000","last_active":"2024-09-13T09:28:51.329759+0000","last_peered":"2024-09-13T09:28:51.329759+0000","last_clean":"2024-09-13T09:28:51.329759+0000","last_became_active":"2024-09-13T09:28:13.679908+0000","last_became_peered":"2024-09-13T09:28:13.679908+0000","last_unstale":"2024-09-13T09:28:51.329759+0000","last_undegraded":"2024-09-13T09:28:51.329759+0000","last_fullsized":"2024-09-13T09:28:51.329759+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:01.428846+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.018367508000000001,"stat_sum":{"num_bytes":18449923,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":10960,"num_read_kb":26455,"num_write":1020,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":15533096,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.12","version":"175'547","reported_seq":5979,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.305439+0000","last_change":"2024-09-13T09:28:51.305876+0000","last_active":"2024-09-13T09:28:51.305439+0000","last_peered":"2024-09-13T09:28:51.305439+0000","last_clean":"2024-09-13T09:28:51.305439+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:28:51.305439+0000","last_undegraded":"2024-09-13T09:28:51.305439+0000","last_fullsized":"2024-09-13T09:28:51.305439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087238699999999997,"stat_sum":{"num_bytes":5376716,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":5606,"num_read_kb":26770,"num_write":660,"num_write_kb":15474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"12","length":"6"}]},{"pgid":"3.13","version":"175'1062","reported_seq":9209,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-13T09:28:51.321756+0000","last_change":"2024-09-13T09:28:51.321961+0000","last_active":"2024-09-13T09:28:51.321756+0000","last_peered":"2024-09-13T09:28:51.321756+0000","last_clean":"2024-09-13T09:28:51.321756+0000","last_became_active":"2024-09-13T09:28:38.185466+0000","last_became_peered":"2024-09-13T09:28:38.185466+0000","last_unstale":"2024-09-13T09:28:51.321756+0000","last_undegraded":"2024-09-13T09:28:51.321756+0000","last_fullsized":"2024-09-13T09:28:51.321756+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:54.167187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010398917000000001,"stat_sum":{"num_bytes":25103583,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":11063,"num_read_kb":35055,"num_write":1348,"num_write_kb":11781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":657,"num_bytes_recovered":28570772,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"12","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":555396600,"num_objects":486,"num_object_clones":73,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":93,"num_read":247576,"num_read_kb":750637,"num_write":30729,"num_write_kb":438878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":385387783,"num_keys_recovered":1247,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22937600,"data_stored":199683503,"data_compressed":11565479,"data_compressed_allocated":15511552,"data_compressed_original":193133016,"omap_allocated":33366,"internal_metadata":0},"log_size":24336,"ondisk_log_size":24336,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50148,"kb_used_data":5080,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321692,"statfs":{"total":96636764160,"available":96585412608,"internally_reserved":0,"allocated":5201920,"data_stored":19504337,"data_compressed":932542,"data_compressed_allocated":1175552,"data_compressed_original":16580107,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.224,"5min":1.256,"15min":1.256},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73799999999999999,"15min":0.77100000000000002},"min":{"1min":0.46999999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91100000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.752,"15min":0.752},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0289999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.622},{"interface":"front","average":{"1min":0.755,"5min":0.78300000000000003,"15min":0.79200000000000004},"min":{"1min":0.39900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.2470000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90700000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.875,"15min":0.89000000000000001},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":1.2649999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1910000000000001,"15min":1.2529999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.484,"5min":8.2249999999999996,"15min":8.2249999999999996},"last":0.85099999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":0.96499999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.01,"15min":1.0549999999999999},"min":{"1min":0.52000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.409,"5min":1.496,"15min":1.496},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98999999999999999,"15min":1.016},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.5009999999999999,"5min":1.573,"15min":1.573},"last":0.997}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39000,"kb_used_data":3788,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332840,"statfs":{"total":96636764160,"available":96596828160,"internally_reserved":0,"allocated":3878912,"data_stored":11113142,"data_compressed":530446,"data_compressed_allocated":716800,"data_compressed_original":9016238,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.84399999999999997,"15min":0.88800000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":3.3069999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.75800000000000001,"15min":0.71799999999999997},"min":{"1min":0.52500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":5.2199999999999998,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":0.98899999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.91500000000000004,"15min":0.96099999999999997},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":3.323,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.501,"5min":0.371,"15min":0.371},"max":{"1min":1.2869999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1970000000000001,"5min":5.149,"15min":5.149},"last":1.018},{"interface":"front","average":{"1min":0.84299999999999997,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.34699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2490000000000001,"5min":4.9829999999999997,"15min":4.9829999999999997},"last":0.96499999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":1.083},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.874}]},{"osd":7,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.91600000000000004,"15min":0.90700000000000003},"min":{"1min":0.41199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.365,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.24,"15min":1.3640000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.0579999999999998,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.83399999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67628,"kb_used_data":4504,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94304212,"statfs":{"total":96636764160,"available":96567513088,"internally_reserved":0,"allocated":4612096,"data_stored":17220338,"data_compressed":1059261,"data_compressed_allocated":1454080,"data_compressed_original":15313496,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.71499999999999997,"15min":0.74399999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.034,"5min":1.462,"15min":1.462},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.78100000000000003,"15min":0.80700000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.091,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.038,"5min":1.992,"15min":1.992},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.78000000000000003,"15min":0.81499999999999995},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1060000000000001,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.77800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77500000000000002,"15min":0.78600000000000003},"min":{"1min":0.34599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.86099999999999999},"min":{"1min":0.42699999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.153,"5min":1.242,"15min":1.242},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:28:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.88500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.80700000000000005,"15min":0.82099999999999995},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.48},{"interface":"front","average":{"1min":0.876,"5min":0.98799999999999999,"15min":1.006},"min":{"1min":0.503,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.393,"5min":1.554,"15min":1.554},"last":1.127}]},{"osd":6,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.91300000000000003,"15min":0.95899999999999996},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.4079999999999999,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.93999999999999995,"15min":0.97699999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5569999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.089}]},{"osd":7,"last update":"Fri Sep 13 09:28:45 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.97599999999999998,"15min":0.98999999999999999},"min":{"1min":0.38400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3049999999999999,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.093},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3660000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.335}]}]},{"osd":4,"up_from":0,"seq":743029342210,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":74,"seq":317827579927,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78312,"kb_used_data":5204,"kb_used_omap":14,"kb_used_meta":73073,"kb_avail":94293528,"statfs":{"total":96636764160,"available":96556572672,"internally_reserved":0,"allocated":5328896,"data_stored":29173723,"data_compressed":1670532,"data_compressed_allocated":2199552,"data_compressed_original":27344896,"omap_allocated":14794,"internal_metadata":74827318},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.496},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.627},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.93600000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.915,"5min":1.915,"15min":1.915},"last":0.878}]},{"osd":7,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.86399999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61524,"kb_used_data":4168,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94310316,"statfs":{"total":96636764160,"available":96573763584,"internally_reserved":0,"allocated":4268032,"data_stored":14559232,"data_compressed":911418,"data_compressed_allocated":1384448,"data_compressed_original":12967367,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.626},"min":{"1min":0.26300000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.97299999999999998,"5min":1.484,"15min":1.484},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.64500000000000002,"15min":0.67300000000000004},"min":{"1min":0.30499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.96599999999999997,"5min":1.335,"15min":1.335},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.97499999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.98199999999999998,"15min":1.0009999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.3420000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.4410000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.94299999999999995,"15min":0.95299999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.306,"5min":1.587,"15min":1.587},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":1.337},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.3049999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.0469999999999999,"15min":1.0920000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.302,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.169},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0049999999999999,"15min":1.038},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1990000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.2}]},{"osd":7,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0409999999999999,"15min":1.0529999999999999},"min":{"1min":0.66900000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.845,"5min":2.048,"15min":2.048},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.111,"15min":1.155},"min":{"1min":0.64000000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.607,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.272}]}]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61884,"kb_used_data":4528,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94309956,"statfs":{"total":96636764160,"available":96573394944,"internally_reserved":0,"allocated":4636672,"data_stored":18555751,"data_compressed":1179017,"data_compressed_allocated":1585152,"data_compressed_original":16801792,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":0.72299999999999998,"15min":0.66900000000000004},"min":{"1min":0.35699999999999998,"5min":0.245,"15min":0.245},"max":{"1min":12.441000000000001,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.5},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68700000000000006,"15min":0.71499999999999997},"min":{"1min":0.32600000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1819999999999999,"5min":1.333,"15min":1.333},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:28:05 2024","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":12.555,"5min":12.555,"15min":12.555},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:27:41 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.93500000000000005,"15min":0.96499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1950000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.052},{"interface":"front","average":{"1min":3.133,"5min":1.4350000000000001,"15min":1.1519999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":49.905999999999999,"5min":49.905999999999999,"15min":49.905999999999999},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.238},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":1.085}]},{"osd":6,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.139,"15min":1.111},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":12.519,"5min":12.519,"15min":12.519},"last":1.196},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.0469999999999999,"15min":1.0760000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.3200000000000001,"5min":1.514,"15min":1.514},"last":1.123}]},{"osd":7,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":1.6870000000000001,"5min":1.194,"15min":1.147},"min":{"1min":0.60799999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":14.143000000000001,"5min":14.143000000000001,"15min":14.143000000000001},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.087,"15min":1.1220000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.0129999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542187,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56888,"kb_used_data":3624,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94314952,"statfs":{"total":96636764160,"available":96578510848,"internally_reserved":0,"allocated":3710976,"data_stored":10153682,"data_compressed":546587,"data_compressed_allocated":827392,"data_compressed_original":8495104,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.61799999999999999},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.37},{"interface":"front","average":{"1min":1.163,"5min":0.78900000000000003,"15min":0.72699999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":8.5869999999999997,"5min":8.5869999999999997,"15min":8.5869999999999997},"last":0.623}]},{"osd":1,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.502},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.73999999999999999,"15min":0.69399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.4820000000000002,"5min":4.4820000000000002,"15min":4.4820000000000002},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:28:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:27:44 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.99399999999999999,"15min":1.008},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.292,"5min":1.996,"15min":1.996},"last":1.02},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0009999999999999,"15min":1.0069999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6379999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.91900000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.88700000000000001,"15min":0.88500000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.131},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.98399999999999999,"15min":1.006},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3720000000000001,"5min":1.651,"15min":1.651},"last":1.081}]},{"osd":6,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.0609999999999999,"15min":1.0069999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":8.8490000000000002,"5min":8.8490000000000002,"15min":8.8490000000000002},"last":1.181},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96399999999999997,"15min":0.97299999999999998},"min":{"1min":0.50900000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:28:49 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.149,"15min":1.1040000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":10.659000000000001,"5min":10.659000000000001,"15min":10.659000000000001},"last":1.099},{"interface":"front","average":{"1min":1.202,"5min":1.04,"15min":1.0149999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":3.9369999999999998,"5min":3.9369999999999998,"15min":3.9369999999999998},"last":1.052}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2142208,"data_stored":15376441,"data_compressed":934030,"data_compressed_allocated":1286144,"data_compressed_original":14704640,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2088960,"data_stored":12952283,"data_compressed":878411,"data_compressed_allocated":1339392,"data_compressed_original":12393927,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1609728,"data_stored":9143365,"data_compressed":546587,"data_compressed_allocated":827392,"data_compressed_original":8495104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":23778307,"data_compressed":1352205,"data_compressed_allocated":1863680,"data_compressed_original":23003136,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7294976,"data_stored":94550550,"data_compressed":5365996,"data_compressed_allocated":6901760,"data_compressed_original":94220288,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2404352,"data_stored":16115050,"data_compressed":1059261,"data_compressed_allocated":1454080,"data_compressed_original":15313496,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1765376,"data_stored":10096561,"data_compressed":530446,"data_compressed_allocated":716800,"data_compressed_original":9016238,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2789376,"data_stored":17670946,"data_compressed":898543,"data_compressed_allocated":1122304,"data_compressed_original":15986187,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:28:55.080 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:28:55.080 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:28:55.080 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 5, 6, 7, 3, 4] 2024-09-13T09:28:55.080 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T09:28:55.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:28:55.081 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:55.073+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:55.139 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.139 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:55.161 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T09:28:55.161 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-13T09:28:55.162 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:28:55.162 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-13T09:28:55.167 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:28:55.168 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-13T09:28:55.168 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:55.163+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:55.255 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.256 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.266 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:55.268 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:55.263+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:55.354 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:28:55.369 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:55.363+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:55.391 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.391 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.401 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:55.470 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:55.463+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:55.505 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.507 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.515 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:55.571 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:55.563+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:55.663 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.663 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.671 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:55.663+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:55.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:55.773 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:55.767+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:55.874 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:55.869+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:55.907 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:55.908 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:55.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:55.974 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:55.971+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:56.025 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.025 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:56.076 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:56.069+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:56.155 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.156 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.166 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:56.177 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:56.169+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:56.187 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:28:56.275 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.275 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.277 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:56.273+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:56.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:56.378 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:56.375+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:56.392 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.393 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.403 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:56.479 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:56.473+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:56.507 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.507 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.517 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:56.579 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:56.573+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:56.629 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.629 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.639 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:56.688 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:56.687+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:56.743 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.743 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.753 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:56.774 INFO:teuthology.orchestra.run.smithi099.stdout:3.d 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.c 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.2 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.1f 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.1c 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.5 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.1d 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.18 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.17 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:3.16 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:1.4 2024-09-13T09:28:56.775 INFO:teuthology.orchestra.run.smithi099.stdout:1.1 2024-09-13T09:28:56.781 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:56.779+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:56.860 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.860 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.870 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:56.917 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:56.915+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:56.932 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:28:56.984 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:56.984 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:56.984 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:56.977+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:56.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:28:57.085 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:57.077+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:57.090 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:57.101 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:57.101 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:57.111 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:28:57.117 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:57.136 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:57.154 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:57.175 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:57.187 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:57.183+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:57.201 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:57.220 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:57.221 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:57.223 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:57.224 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:57.229 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:57.229 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:57.230 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:57.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:28:57.287 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:57.281+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:57.341 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:57.341 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:57.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:28:57.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.1c --file /var/log/ceph/exp.3.1c.5 2024-09-13T09:28:57.387 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:57.381+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:57.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029993755, 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:28:57.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029978698999997, 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:28:57.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:57.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029969291, 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:57.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:28:57.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029954998000001, 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:57.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:57.476 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029869292000001, 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:57.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:28:57.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:28:57.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:57.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029513043000001, 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:57.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:28:57.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029466206999999, 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:57.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:28:57.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029427220999999, 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:28:57.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029407077000002, 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:28:57.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:28:57.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029388009000002, 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.500 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:28:57.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029345919999997, 2024-09-13T09:28:57.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:28:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029232417999999, 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.509 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:28:57.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.513 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029214238999998, 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:28:57.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029204493999998, 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:57.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029194306999997, 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:28:57.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.029101441000002, 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:28:57.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.028998029, 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:28:57.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:28:57.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.028676150999999, 2024-09-13T09:28:57.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:28:57.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.028260209999999, 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.541 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:28:57.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:57.545 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.028006499999996, 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:57.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:28:57.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:28:57.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:28:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.550 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:57.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:28:57.552 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:57.545+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:57.652 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:57.651+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:57.709 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:57.709 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:57.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:28:57.753 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:57.751+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:57.834 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:57.835 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:57.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:28:57.854 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:28:57.851+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:28:57.949 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:57.949 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:57.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:57.949 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:28:57.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 194.98465157699999, 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:57.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.954 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.871431833, 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:28:57.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:28:57.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.871033818, 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:28:57.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.195926942, 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:28:57.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:28:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:28:57.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.19591201, 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:57.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:57.974 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.19587890400001, 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:28:57.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.195862611, 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:57.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:28:57.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.195846496, 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.195815455, 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:28:57.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.19578383199999, 2024-09-13T09:28:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:28:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.18283577099999, 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:57.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:28:57.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:57.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:57.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.000 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.182818715, 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:28:58.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.18280254699999, 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:28:58.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:58.009 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.182588885, 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:28:58.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:28:58.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.18256957400001, 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:28:58.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.182553181, 2024-09-13T09:28:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:28:58.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:28:58.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.18253751899999, 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:28:58.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 131.18251116499999, 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:58.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.031 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.653998438000002, 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:28:58.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:28:58.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 79.542084322999997, 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:28:58.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:28:58.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:28:58.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:28:58.044 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:58.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:28:58.046 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:58.043+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:58.147 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:58.143+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:58.165 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:58.165 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:58.165 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:58.165 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:58.175 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:28:58.200 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:28:58.248 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:58.247+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:58.291 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:58.291 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:58.291 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:58.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:28:58.349 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:28:58.341+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:28:58.406 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 113.665355101, 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:28:58.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:28:58.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:28:58.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:28:58.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:28:58.414 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.423182349999998, 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.415 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:28:58.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:28:58.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:58.419 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.423159134999999, 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:58.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.421 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:28:58.423 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.419030818000003, 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.424 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:28:58.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.418978258000003, 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.429 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:28:58.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.418941562999997, 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:28:58.433 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:28:58.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.437 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.418921933, 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.438 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:28:58.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:28:58.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.418899754000002, 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:28:58.442 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:28:58.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:28:58.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.415143970999999, 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:28:58.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:28:58.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.415118636999999, 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.451 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.415100799999998, 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.455 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.458 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.459 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.415088415, 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:58.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.461 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:28:58.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.415059653999997, 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.464 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:28:58.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.410907442999999, 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.469 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:28:58.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.410879155000003, 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:28:58.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.474 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:28:58.475 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:28:58.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.410853021000001, 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.478 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:28:58.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.480 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.410828383999998, 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:28:58.482 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:28:58.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.486 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.487 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.410745810000002, 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:28:58.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:28:58.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.410702907000001, 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.492 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:28:58.493 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.407077184999999, 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:28:58.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:58.500 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:58.501 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:58.502 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:28:58.503 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:58.497+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:58.509 INFO:teuthology.orchestra.run.smithi099.stderr:Exporting 3.1c info 3.1c( v 179'776 (0'0,179'776] local-lis/les=136/137 n=14 ec=21/21 lis/c=136/136 les/c/f=137/137/0 sis=136) 2024-09-13T09:28:58.534 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3831950c:::rbd_directory:head# 2024-09-13T09:28:58.535 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:38abe4f3:::rbd_data.1118474c503.0000000000000006:6# 2024-09-13T09:28:58.544 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:38abe4f3:::rbd_data.1118474c503.0000000000000006:head# 2024-09-13T09:28:58.547 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:38aeb2f6:::rbd_object_map.1118474c503:head# 2024-09-13T09:28:58.547 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3985481d:::rbd_data.111851704b2b.0000000000000016:9# 2024-09-13T09:28:58.549 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3985481d:::rbd_data.111851704b2b.0000000000000016:head# 2024-09-13T09:28:58.550 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3a4b0e7d:::rbd_data.1118a28cd5c4.00000000000000d7:head# 2024-09-13T09:28:58.550 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3b3efbd2:::rbd_data.1118a28cd5c4.0000000000000027:head# 2024-09-13T09:28:58.551 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3b833fa2:::rbd_data.11186f48fbfa.0000000000000052:head# 2024-09-13T09:28:58.551 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3c6dc773:::rbd_data.1118e98b86fc.000000000000000a:d# 2024-09-13T09:28:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3c6dc773:::rbd_data.1118e98b86fc.000000000000000a:head# 2024-09-13T09:28:58.553 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3d9779f3:::rbd_data.1118a28cd5c4.0000000000000067:head# 2024-09-13T09:28:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3f0f4d30:::rbd_data.111888f95246.0000000000000010:head# 2024-09-13T09:28:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:3fda0657:::rbd_object_map.111867511a81:head# 2024-09-13T09:28:58.554 INFO:teuthology.orchestra.run.smithi099.stderr:Export successful 2024-09-13T09:28:58.565 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:28:58.603 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:58.599+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:58.618 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:58.618 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:58.618 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:28:58.618 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:58.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:28:58.704 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:58.703+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:58.780 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:58.780 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:28:58.780 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:28:58.780 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:28:58.780 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:58.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:28:58.805 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:28:58.803+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:28:58.896 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:28:58.896 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:28:58.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:28:58.896 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [] 2024-09-13T09:28:58.896 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:28:58.905 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:58.901+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:58.906 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:28:59.007 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:59.005+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:59.022 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:59.022 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:59.022 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:28:59.022 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:59.033 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:28:59.107 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:59.101+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:59.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.1c 2024-09-13T09:28:59.142 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:28:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:28:59.142 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:28:59.142 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:59.154 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:28:59.208 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:59.207+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:59.225 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:28:59.268 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 137.702517199, 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.269 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:28:59.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:28:59.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 105.11485881999999, 2024-09-13T09:28:59.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:28:59.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.984566694999998, 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:28:59.279 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:28:59.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:28:59.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 52.400356244000001, 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.284 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:28:59.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.999022611999997, 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.289 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:28:59.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.999004604, 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.293 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.294 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:28:59.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:28:59.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998987264999997, 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:28:59.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:28:59.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:28:59.302 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998970008000001, 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.307 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998952471999999, 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:28:59.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.312 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998935402000001, 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998921742999997, 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:28:59.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:28:59.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.321 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998912070000003, 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.322 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:28:59.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:59.326 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998897524999997, 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:28:59.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998880858, 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:28:59.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.335 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.998863767000003, 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:28:59.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.997008190999999, 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.340 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:28:59.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:28:59.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.996992083000002, 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:28:59.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:28:59.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.349 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.996977719, 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:28:59.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.996963891, 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:28:59.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.358 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 51.996946190999999, 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:28:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:28:59.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:28:59.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:28:59.364 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:28:59.364 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:28:59.365 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:28:59.366 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:59.363+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:59.368 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:28:59.392 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:28:59.416 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:28:59.436 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:28:59.459 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:28:59.466 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:59.463+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:59.484 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:59.484 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:59.491 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:28:59.494 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:28:59.513 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:28:59.513 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:28:59.513 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:28:59.513 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:28:59.513 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:28:59.514 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:28:59.514 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:28:59.514 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:28:59.514 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:28:59.516 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:28:59.516 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:28:59.567 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:28:59.563+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:28:59.620 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:59.621 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:59.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:28:59.667 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:59.661+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:59.736 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:59.737 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:59.746 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:28:59.768 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:28:59.765+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:28:59.858 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:59.859 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:59.869 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:28:59.869 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:28:59.867+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:28:59.970 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:28:59.969+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:28:59.978 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:28:59.978 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:28:59.989 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:00.071 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:00.065+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:00.098 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.098 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:00.171 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:00.167+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:00.223 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.224 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:00.272 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:00.271+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:00.337 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.337 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:00.370 INFO:teuthology.orchestra.run.smithi099.stdout: marking collection for removal 2024-09-13T09:29:00.371 INFO:teuthology.orchestra.run.smithi099.stdout:setting '_remove' omap key 2024-09-13T09:29:00.371 INFO:teuthology.orchestra.run.smithi099.stdout:finish_remove_pgs 3.1c_head removing 3.1c 2024-09-13T09:29:00.373 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:00.365+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:00.376 INFO:teuthology.orchestra.run.smithi099.stdout:Remove successful 2024-09-13T09:29:00.452 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.453 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:00.474 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:00.471+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:00.566 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.566 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:00.576 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:00.569+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:00.579 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:29:00.676 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:00.675+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:00.729 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.730 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:00.777 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:00.775+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:00.844 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.844 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.853 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:00.878 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:00.873+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:00.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1c.5 2024-09-13T09:29:00.961 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:00.962 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:00.972 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:00.978 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:00.975+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:01.076 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:01.076 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:01.080 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:01.075+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:01.086 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:01.181 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:01.177+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:01.191 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:01.192 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:01.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:01.282 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:01.279+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:01.320 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:01.320 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:01.330 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:01.383 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:01.377+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:01.449 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:01.449 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:01.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:01.484 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:01.483+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:01.517 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:01.565 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:01.565 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:01.574 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:01.585 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:01.581+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:29:01.673 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:01.683 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:01.683 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:01.683 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:01.683 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:01.685 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:01.683+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:01.692 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:01.693 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:01.713 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:01.735 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:01.757 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:01.780 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:01.786 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:01.781+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:01.798 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:01.798 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:01.798 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:01.798 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:01.798 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:01.802 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:01.803 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:01.807 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:01.808 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:01.809 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:01.887 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:01.883+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:01.964 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:01.964 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 140.39971018200001, 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:01.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:01.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.969 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.812051803, 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.970 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:01.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.681759677999999, 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.975 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:01.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.097549227000002, 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:01.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696215594999998, 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:01.984 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:01.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:01.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696197587, 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.989 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:01.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696180247999997, 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:01.993 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:01.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:01.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.997 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696162991000001, 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:01.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:01.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:02.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.002 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696145455, 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:02.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696128385000002, 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:02.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696114725999998, 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.012 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:02.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:02.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:02.016 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696105052999997, 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:02.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696090507999997, 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:02.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:02.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.025 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696073841, 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:02.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.696056749999997, 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:02.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:02.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:02.037 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.694201174, 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.038 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:02.043 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.694185066000003, 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:02.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:02.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:02.048 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.694170702000001, 2024-09-13T09:29:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:02.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:02.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.694156874000001, 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:02.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:02.058 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 54.694139174, 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.063 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.063 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:02.064 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:02.065 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:02.061+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:02.166 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:02.161+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:02.168 INFO:teuthology.orchestra.run.smithi099.stdout:get_pg_num_history pg_num_history pg_num_history(e179 pg_nums {1={16=8},2={17=1},3={21=32}} deleted_pools ) 2024-09-13T09:29:02.168 INFO:teuthology.orchestra.run.smithi099.stdout:Importing pgid 3.1c 2024-09-13T09:29:02.170 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3831950c:::rbd_directory:head# 2024-09-13T09:29:02.170 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:29:02.171 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:38abe4f3:::rbd_data.1118474c503.0000000000000006:6# 2024-09-13T09:29:02.181 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:02.181 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:02.181 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:02.188 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:38abe4f3:::rbd_data.1118474c503.0000000000000006:head# 2024-09-13T09:29:02.189 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 6=[]:{6=[6]} 2024-09-13T09:29:02.192 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:02.193 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:38aeb2f6:::rbd_object_map.1118474c503:head# 2024-09-13T09:29:02.193 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:29:02.194 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3985481d:::rbd_data.111851704b2b.0000000000000016:9# 2024-09-13T09:29:02.198 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3985481d:::rbd_data.111851704b2b.0000000000000016:head# 2024-09-13T09:29:02.199 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 9=[]:{9=[9]} 2024-09-13T09:29:02.199 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3a4b0e7d:::rbd_data.1118a28cd5c4.00000000000000d7:head# 2024-09-13T09:29:02.200 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:29:02.202 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3b3efbd2:::rbd_data.1118a28cd5c4.0000000000000027:head# 2024-09-13T09:29:02.202 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:29:02.203 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3b833fa2:::rbd_data.11186f48fbfa.0000000000000052:head# 2024-09-13T09:29:02.204 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 16=[]:{} 2024-09-13T09:29:02.204 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3c6dc773:::rbd_data.1118e98b86fc.000000000000000a:d# 2024-09-13T09:29:02.206 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3c6dc773:::rbd_data.1118e98b86fc.000000000000000a:head# 2024-09-13T09:29:02.207 INFO:teuthology.orchestra.run.smithi099.stdout:snapset d=[]:{d=[d]} 2024-09-13T09:29:02.207 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3d9779f3:::rbd_data.1118a28cd5c4.0000000000000067:head# 2024-09-13T09:29:02.207 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:29:02.208 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3f0f4d30:::rbd_data.111888f95246.0000000000000010:head# 2024-09-13T09:29:02.208 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 2e=[]:{} 2024-09-13T09:29:02.209 INFO:teuthology.orchestra.run.smithi099.stdout:Write #3:3fda0657:::rbd_object_map.111867511a81:head# 2024-09-13T09:29:02.209 INFO:teuthology.orchestra.run.smithi099.stdout:snapset 0=[]:{} 2024-09-13T09:29:02.209 INFO:teuthology.orchestra.run.smithi099.stdout:write_pg epoch 179 info 3.1c( v 179'776 (0'0,179'776] local-lis/les=136/137 n=14 ec=21/21 lis/c=136/136 les/c/f=137/137/0 sis=136) 2024-09-13T09:29:02.214 INFO:teuthology.orchestra.run.smithi099.stdout:Import successful 2024-09-13T09:29:02.266 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:02.263+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:02.302 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:02.302 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:02.303 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:02.303 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:02.303 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:02.313 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:02.367 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:02.361+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:02.423 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:02.423 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842767812000005, 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.424 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:02.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:02.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842755574999998, 2024-09-13T09:29:02.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.430 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:02.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842740178, 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.434 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:02.435 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.436 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.437 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842725302999995, 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.439 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.440 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:02.441 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.442 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842708094000002, 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.443 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.444 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:02.445 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.446 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:02.447 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842687639000005, 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.448 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.449 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:02.450 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.451 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:02.452 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842669559000001, 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.453 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.454 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.455 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:02.456 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842653182999996, 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:02.457 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.458 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.459 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:02.460 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.461 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:02.462 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.463 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:02.464 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.465 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842637831999994, 2024-09-13T09:29:02.466 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:02.467 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.468 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.469 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.470 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:02.471 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.472 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842621128000005, 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.473 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.474 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.475 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.476 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:02.477 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.478 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.479 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842603045999994, 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.480 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:02.481 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:02.482 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.483 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.484 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:02.485 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842586300999997, 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.486 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.487 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.488 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.489 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.490 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.491 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842541642, 2024-09-13T09:29:02.492 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.493 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.494 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:02.495 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.496 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.497 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.498 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.839592202000006, 2024-09-13T09:29:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:02.499 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.500 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.501 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:02.502 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:02.503 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:02.504 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.839571664000005, 2024-09-13T09:29:02.505 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.506 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:02.507 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:02.508 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.509 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:02.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:02.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:02.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:02.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:02.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.516 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.839549048999999, 2024-09-13T09:29:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.517 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.518 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:02.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:02.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.839530181000001, 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:02.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:02.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:02.531 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.839495587000002, 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:02.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:02.537 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.835216681999995, 2024-09-13T09:29:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.538 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:02.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:02.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.544 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.835152710000003, 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:02.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:02.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:02.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.550 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.551 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.551 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.551 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.551 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:02.552 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:02.553 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:02.545+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:02.654 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:02.651+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:02.675 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:02.676 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:02.676 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:02.676 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:02.686 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:29:02.755 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:02.751+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:02.783 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /var/log/ceph/exp.3.1c.5 2024-09-13T09:29:02.790 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:02.791 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:02.791 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:02.791 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:02.791 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:02.801 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:02.856 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:02.851+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:29:02.913 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.350689596000002, 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.915 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:29:02.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:29:02.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.722322022, 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:29:02.919 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:29:02.923 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.680743428, 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.924 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.926 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.927 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:29:02.928 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.679328249000001, 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.929 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.930 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.931 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:29:02.932 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.933 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.679309596000003, 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.934 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.935 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:29:02.936 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.937 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.679237047999997, 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.938 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.939 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:29:02.940 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.941 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.942 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.943 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:29:02.944 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.679000273, 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.945 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.946 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.947 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:29:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:29:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.948 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:29:02.949 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.950 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:29:02.951 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.952 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:29:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678957441999998, 2024-09-13T09:29:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:29:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.953 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.954 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:29:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.955 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.956 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.957 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:29:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:29:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.958 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:29:02.959 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:29:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:29:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.960 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:29:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:29:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.961 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678933780000001, 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:29:02.962 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.963 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.964 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.965 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.966 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.967 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.968 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678723749, 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.969 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.970 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:29:02.971 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.972 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.973 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:29:02.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.975 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678696877999997, 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.976 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678678984000001, 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:29:02.981 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:29:02.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:29:02.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:29:02.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678662737000003, 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:29:02.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:02.992 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678564672999997, 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:02.993 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:02.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:29:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:02.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:29:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:29:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:02.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678331438000001, 2024-09-13T09:29:03.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:03.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:29:03.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:29:03.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:29:03.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:03.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678236831, 2024-09-13T09:29:03.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:03.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:29:03.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:29:03.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:29:03.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678211306000001, 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:03.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:29:03.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:29:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:29:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:03.016 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.018 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.678188126999999, 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:03.019 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:03.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:03.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:29:03.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:29:03.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:29:03.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:03.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:29:03.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:29:03.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.677914104000003, 2024-09-13T09:29:03.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:29:03.028 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:03.028 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:03.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:03.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:03.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:29:03.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:03.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 55.677729952999996, 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:03.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:03.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:29:03.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:29:03.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.040 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:03.041 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:03.041 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:03.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:03.043 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:03.039+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:03.143 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:03.137+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:03.159 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:03.159 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:03.159 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:03.169 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:03.243 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:03.237+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:03.318 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:03.318 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:03.318 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:03.318 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:03.318 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:03.328 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:03.344 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:03.343+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:03.435 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007557767000002, 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007542710999999, 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007533303000002, 2024-09-13T09:29:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007519010000003, 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:03.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007433304000003, 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:03.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:03.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:03.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007077055000003, 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:03.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:03.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.007030219000001, 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:03.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:03.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:03.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006991233000001, 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:03.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:03.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006971088999997, 2024-09-13T09:29:03.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:03.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:03.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006952020999996, 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:03.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006909931999999, 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:03.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006796430000001, 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:03.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006778251, 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:03.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:03.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:03.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006768506, 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:03.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:03.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006758318999999, 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:03.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006665452999997, 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.505 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:03.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:03.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006562041000002, 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:03.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:03.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.006240163000001, 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:03.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:03.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.005824222000001, 2024-09-13T09:29:03.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:03.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 57.005570511999998, 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:03.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.527 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:03.528 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:03.529 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:03.527+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:03.605 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:29:03.638 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:03.638 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:03.638 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:03.648 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:03.652 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:03.651+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:03.730 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:03.725+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:03.763 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:03.763 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:03.763 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:03.763 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:03.764 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:03.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:03.809 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:03.831 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:03.827+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:03.925 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:03.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 200.96052791, 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:03.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:03.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.847308166, 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.931 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:03.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:03.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.846910151, 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:03.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:03.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:03.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.171803275, 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:03.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:03.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.945 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.171788343, 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:03.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.17175523700001, 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:03.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.171738944, 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:03.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:03.959 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.171722829, 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:03.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:03.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.171691788, 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.968 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.17166016499999, 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:03.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.15871210399999, 2024-09-13T09:29:03.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.158695048, 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.15867888, 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:03.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.158465218, 2024-09-13T09:29:03.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:03.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.15844590699999, 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:03.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:03.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.15842951400001, 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.997 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:03.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:03.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.158413852, 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:04.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.158387498, 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:04.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:04.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 90.629874771000004, 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:04.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:04.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:04.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:04.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 85.517960656, 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:04.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:04.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:04.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:04.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:04.024 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:04.025 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:04.025 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:04.025 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:04.025 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:04.026 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:04.026 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:04.017+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:04.034 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:04.058 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:04.081 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:04.081 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:04.082 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:04.085 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:04.086 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:04.128 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:04.127+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:04.135 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.135 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.145 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:04.226 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:04.221+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:04.251 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.251 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.261 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:04.327 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:04.325+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:04.364 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.365 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:04.427 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:04.423+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:04.490 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.490 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:04.528 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:04.521+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:04.592 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:29:04.609 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.609 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:04.629 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:04.625+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:04.722 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.722 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.730 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:04.727+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:04.732 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:04.831 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:04.827+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:04.850 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.850 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:04.932 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:04.929+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:04.970 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:04.970 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:04.980 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:05.033 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:05.029+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:05.090 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.090 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:05.134 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:05.129+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:05.209 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.209 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.219 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:05.235 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:05.231+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:05.329 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.330 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.336 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:05.335+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:05.339 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:05.437 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:05.435+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:05.492 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.492 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.502 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:05.538 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:05.535+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:05.606 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.606 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.616 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:29:05.639 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:05.637+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:05.740 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:05.737+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:05.769 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.770 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:05.842 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:05.839+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:05.892 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:05.892 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:05.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:05.946 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:05.941+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:06.019 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:06.019 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:06.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:06.043 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:06.037+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:06.087 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:06.136 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:06.136 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:06.144 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:06.143+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:06.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:06.232 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:06.245 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:06.243+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:06.250 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:06.272 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:06.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:06.296 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:06.296 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:06.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:06.315 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:06.344 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:06.346 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:06.343+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:06.373 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:06.374 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:06.379 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:06.379 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:06.416 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:06.416 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:06.416 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:06.416 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:06.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:06.447 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:06.443+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:06.525 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:06.526 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:06.526 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:06.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:06.547 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:06.545+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:06.605 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:29:06.649 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:06.645+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:06.652 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 203.689358078, 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:06.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 140.57613833400001, 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:06.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 140.575740319, 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:06.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:06.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:06.667 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.900633443, 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:06.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:06.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.900618511, 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.676 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.90058540499999, 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:06.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:06.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.900569112, 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.681 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:06.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:06.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.90055299700001, 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:06.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:06.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.90052195600001, 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:06.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.90049033299999, 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.88754227199999, 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.699 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:06.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.887525216, 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:06.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:06.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.708 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.887509048, 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:06.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.712 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.88729538600001, 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:06.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:06.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.88727607499999, 2024-09-13T09:29:06.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:06.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.721 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.88725968200001, 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.88724402, 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:06.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:06.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 139.887217666, 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:06.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:06.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:06.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.358704939000006, 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:06.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:06.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:06.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.246790824000001, 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:06.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:06.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:06.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:06.750 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:06.752 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:06.752 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:06.749+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:06.852 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:06.851+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:06.874 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:06.874 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:06.874 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:06.874 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:06.884 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:06.954 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:06.951+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:06.988 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:06.988 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:06.988 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:06.988 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:06.989 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:06.997 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:07.055 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:07.053+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:07.156 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 122.415094512, 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.157 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:07.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:07.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.172921760999998, 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.164 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:07.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.168 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.172898545999999, 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.169 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:07.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.168770229000003, 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.174 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:07.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.178 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.168717669000003, 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:07.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:07.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.168680973999997, 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.183 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:07.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.187 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.168661344, 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:07.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.192 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.168639165000002, 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:07.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.164883381999999, 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:07.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:07.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.164858047999999, 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:07.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:07.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.164840210999998, 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.207 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.164827826, 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:07.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.215 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.164799064999997, 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.216 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:07.220 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.160646853999999, 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:07.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.160618565999997, 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.160592432000001, 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:07.236 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.160567794999999, 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.237 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:07.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:07.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.160485221000002, 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.242 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:07.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.246 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.160442318000001, 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:07.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.156816595999999, 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:07.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:07.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:07.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:07.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:07.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:07.257 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:07.258 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:07.155+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:07.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:07.260 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:07.259+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:07.362 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:07.361+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:07.376 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:07.376 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:07.387 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:07.463 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:07.461+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:07.540 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:07.540 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:07.540 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:07.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:07.564 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:07.563+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:07.621 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:29:07.653 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:07.653 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:07.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:07.654 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [] 2024-09-13T09:29:07.654 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:07.663 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:07.677 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:07.675+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:07.765 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:07.765+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:07.789 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:07.789 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:07.789 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:07.799 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:07.847 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0, 2] out_osds: [4] dead_osds: [5] live_osds: [0, 1, 2, 6, 7, 3, 4] 2024-09-13T09:29:07.847 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:29:07.848 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-13T09:29:07.848 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-13T09:29:07.892 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:07.891+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:07.909 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:07.909 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:07.910 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:07.910 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:07.910 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:07.920 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:07.967 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:07.963+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:08.027 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:08.027 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 146.46133740400001, 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:08.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.033 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 113.873679025, 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:08.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:08.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:08.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.743386899999997, 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:08.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:08.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 61.159176449, 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.045 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757842816999997, 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:08.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.054 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757824808999999, 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.055 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:08.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:08.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.059 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757807470000003, 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.064 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757790213, 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:08.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:08.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:08.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.069 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757772676999998, 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:08.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.074 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757755607, 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:08.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:08.079 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757741948000003, 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:08.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.083 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757732275000002, 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:08.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:08.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757717730000003, 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:08.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:08.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757701062999999, 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:08.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.757683972000002, 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:08.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:08.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.755828395999998, 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.102 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.755812288000001, 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.106 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.110 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.755797923999999, 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:08.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:08.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.755784095999999, 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:08.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:08.119 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 60.755766395999999, 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:08.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:08.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:08.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:08.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:08.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:08.124 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:08.125 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:08.126 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:08.125+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:08.150 INFO:tasks.ceph.mon.a.smithi063.stderr:2024-09-13T09:29:08.147+0000 7f00df3a9700 -1 mon.a@0(leader).osd e189 definitely_dead 0 2024-09-13T09:29:08.226 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:08.223+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:08.259 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:08.259 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:08.259 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:08.259 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:08.269 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:08.326 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:08.323+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:08.377 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:08.377 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:08.377 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:08.377 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:08.377 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:08.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:08.388 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:08.436 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:08.435+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:08.528 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:08.527+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:08.539 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:08.543 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:08.543 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:08.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:08.560 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:08.583 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:08.605 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:08.633 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:08.634 INFO:teuthology.orchestra.run.smithi063.stderr:marked in osd.4. 2024-09-13T09:29:08.636 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:08.629+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:08.652 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-13T09:29:08.662 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:08.662 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:08.663 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:08.665 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:08.665 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:08.671 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:08.671 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:08.681 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:29:08.730 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:08.729+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:08.833 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:08.831+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:08.835 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:08.835 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:08.845 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:08.934 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:08.931+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:08.953 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:08.953 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:08.963 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:09.037 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:09.035+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:09.077 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.077 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:09.138 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:09.135+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:09.197 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.197 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:09.240 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:09.237+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:09.313 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.313 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:09.341 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:09.339+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:09.429 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.429 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:09.444 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:09.441+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:09.542 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:09.541+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:09.546 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.547 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.556 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:09.634 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:29:09.643 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:09.641+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:09.711 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.711 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.720 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:09.744 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:09.741+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:09.835 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.836 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.846 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:09.846 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:09.843+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:09.945 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:09.945 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:09.946 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:09.945+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:09.954 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:10.046 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:10.045+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:10.062 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.062 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:10.148 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:10.145+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:10.186 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.186 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.196 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:10.249 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:10.247+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:10.299 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.300 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.309 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:10.350 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:10.347+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:10.418 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.418 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.428 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:10.473 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:10.471+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:10.546 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.546 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.551 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:10.547+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:10.556 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:10.652 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:10.651+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:29:10.666 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:10.676 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.677 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.686 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:10.753 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:10.753+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:10.797 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:10.798 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:10.807 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:10.813 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:10.833 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:10.854 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:10.851+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:10.868 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:10.940 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:10.955 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:10.951+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:10.956 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:10.956 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:10.957 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:10.957 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:10.970 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:10.974 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:10.999 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:10.999 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:10.999 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:10.999 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:10.999 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:11.000 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:11.000 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:11.000 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:11.000 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:11.003 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:11.003 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:11.056 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:11.055+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:11.077 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:11.077 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:11.077 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:11.077 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:11.077 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:11.087 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:11.158 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:11.155+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:11.195 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615357939999996, 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:11.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615345703000003, 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.201 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:11.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615330306000004, 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:11.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615315430999999, 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:11.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.214 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615298222000007, 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:11.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.219 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615277766999995, 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:11.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615259687000005, 2024-09-13T09:29:11.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:11.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.226 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:11.228 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615243311, 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.229 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.231 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:11.233 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615227959999999, 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.234 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615211255999995, 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:11.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:11.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615193173999998, 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:11.243 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:11.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615176429000002, 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.248 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:11.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.252 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.615131770000005, 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:11.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:11.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:11.257 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.612182329999996, 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:11.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.259 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:11.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.612161791999995, 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.262 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.612139177000003, 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:11.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:11.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.612120309000005, 2024-09-13T09:29:11.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:11.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.612085715000006, 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.276 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.277 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.60780681, 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.281 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:11.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:11.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 83.607742837999993, 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:11.285 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:11.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.290 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.290 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:11.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:11.291 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:11.287+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:11.392 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:11.389+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:11.458 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:11.458 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:11.458 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:11.458 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:11.469 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:29:11.493 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:11.491+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:11.577 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:11.577 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:11.577 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:11.577 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:11.577 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:11.587 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:11.593 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:11.591+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:11.696 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:11.691+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:166522 3.18 3.96576298 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e127)", 2024-09-13T09:29:11.709 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.147673026000007, 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99932598299999997, 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 166522 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558460+0000", 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558457+0000", 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558486+0000", 2024-09-13T09:29:11.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8909999999999999e-05 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558513+0000", 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7492e-05 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558529+0000", 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5199e-05 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558611+0000", 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2672999999999999e-05 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:01.558627+0000", 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5517999999999999e-05 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557765+0000", 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.99913844399999996 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.713 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:02.557786+0000", 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0307999999999999e-05 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:180764 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2082816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.519305451999998, 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055755458000000001, 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.714 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 180764 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:11.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186827+0000", 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186830+0000", 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186837+0000", 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.7230000000000004e-06 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.186843+0000", 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237714+0000", 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050871138000000003 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.237789+0000", 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5449000000000002e-05 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.242583+0000", 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0047934459999999998 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181665 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1353728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:29:11.718 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.477726857999997, 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.067079618999999993, 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181665 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228406+0000", 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228405+0000", 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228408+0000", 2024-09-13T09:29:11.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.463e-06 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228418+0000", 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8830000000000001e-06 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.228423+0000", 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2619999999999998e-06 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279611+0000", 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.051188286999999999 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.279755+0000", 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014419199999999999 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.295485+0000", 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.015729844 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181700 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1763328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.476311679000005, 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053684935000000003, 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.723 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181700 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229821+0000", 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229820+0000", 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229823+0000", 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229830+0000", 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:29:11.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229835+0000", 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0329999999999999e-06 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283290+0000", 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053455137 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283393+0000", 2024-09-13T09:29:11.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103408 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283506+0000", 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000112359 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181702 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1783808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.476293025999993, 2024-09-13T09:29:11.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053911727, 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181702 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229840+0000", 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229839+0000", 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229842+0000", 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7939999999999998e-06 2024-09-13T09:29:11.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229849+0000", 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9859999999999997e-06 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229854+0000", 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8280000000000002e-06 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283530+0000", 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053676441999999998 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283640+0000", 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011006 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283751+0000", 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110998 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.731 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181704 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1804288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.476220478000002, 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054083085000000003, 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181704 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.732 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229912+0000", 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229911+0000", 2024-09-13T09:29:11.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229916+0000", 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2870000000000004e-06 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229925+0000", 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6670000000000002e-06 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.229931+0000", 2024-09-13T09:29:11.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283778+0000", 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053846317999999997 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283881+0000", 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103114 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.283995+0000", 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114392 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181706 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1824768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475983702999997, 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054066901000000001, 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181706 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230149+0000", 2024-09-13T09:29:11.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230148+0000", 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230154+0000", 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230174+0000", 2024-09-13T09:29:11.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0253000000000001e-05 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230185+0000", 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0634e-05 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284016+0000", 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.053831760999999999 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284122+0000", 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105549 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284216+0000", 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3714000000000004e-05 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181708 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1845248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.740 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475940871999995, 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054149164, 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181708 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230192+0000", 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230191+0000", 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230195+0000", 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3790000000000002e-06 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230204+0000", 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6849999999999996e-06 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230210+0000", 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.913e-06 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284230+0000", 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054019709999999999 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284270+0000", 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.0512000000000001e-05 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284341+0000", 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0421999999999998e-05 2024-09-13T09:29:11.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181710 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1865728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475917210000006, 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054229039, 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.745 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181710 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230215+0000", 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230218+0000", 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8550000000000002e-06 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230224+0000", 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0730000000000001e-06 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230229+0000", 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284353+0000", 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054124463999999997 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284391+0000", 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7907999999999997e-05 2024-09-13T09:29:11.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284444+0000", 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3217999999999998e-05 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181712 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1886208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475707178999997, 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054230038000000001, 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.749 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181712 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230425+0000", 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230424+0000", 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230430+0000", 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6710000000000004e-06 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230439+0000", 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230446+0000", 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.063e-06 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284465+0000", 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054018654999999999 2024-09-13T09:29:11.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284538+0000", 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3555999999999998e-05 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284655+0000", 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011724 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181714 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1906688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475680307999994, 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054423523000000001, 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181714 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230452+0000", 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230455+0000", 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230461+0000", 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2310000000000001e-06 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230466+0000", 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284688+0000", 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054222054999999998 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284762+0000", 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3961999999999994e-05 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284876+0000", 2024-09-13T09:29:11.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113659 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181715 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1927168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475662413999999, 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054615428000000001, 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.758 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181715 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230470+0000", 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230472+0000", 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230478+0000", 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6459999999999998e-06 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230482+0000", 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4089999999999996e-06 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284899+0000", 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054416898999999998 2024-09-13T09:29:11.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.284975+0000", 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5631999999999997e-05 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285086+0000", 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000110956 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.762 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181716 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1947648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475646166999994, 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054808345000000001, 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181716 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230486+0000", 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230488+0000", 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4439999999999998e-06 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230494+0000", 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2070000000000003e-06 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230499+0000", 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8579999999999999e-06 2024-09-13T09:29:11.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285108+0000", 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054609106999999997 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285181+0000", 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2941000000000005e-05 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285295+0000", 2024-09-13T09:29:11.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00011413000000000001 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181718 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1968128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475548102999994, 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054949308000000002, 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.767 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181718 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230584+0000", 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230583+0000", 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230588+0000", 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9069999999999998e-06 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230598+0000", 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1819999999999995e-06 2024-09-13T09:29:11.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230605+0000", 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0720000000000002e-06 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285321+0000", 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054715936999999999 2024-09-13T09:29:11.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285403+0000", 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2102999999999996e-05 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285534+0000", 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013105699999999999 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181720 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475314867999998, 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054948065999999997, 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181720 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230818+0000", 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230817+0000", 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230822+0000", 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.429e-06 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230832+0000", 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7680000000000003e-06 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230839+0000", 2024-09-13T09:29:11.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8889999999999996e-06 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285564+0000", 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054724636 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285660+0000", 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6564999999999994e-05 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285766+0000", 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105622 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181722 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2009088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475220261000004, 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055071237000000002, 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.776 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181722 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230912+0000", 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230916+0000", 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2819999999999998e-06 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230925+0000", 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6240000000000008e-06 2024-09-13T09:29:11.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230931+0000", 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285791+0000", 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.054860158999999999 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285879+0000", 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8274000000000002e-05 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.285984+0000", 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104114 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181724 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2029568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:29:11.780 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475194736000006, 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055261167, 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181724 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230938+0000", 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230937+0000", 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230940+0000", 2024-09-13T09:29:11.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.852e-06 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230947+0000", 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230952+0000", 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1549999999999998e-06 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286009+0000", 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055057547999999998 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286095+0000", 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6119999999999995e-05 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286199+0000", 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010352 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.784 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181726 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2050048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.475171556999996, 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055442783000000002, 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181726 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.785 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230961+0000", 2024-09-13T09:29:11.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230963+0000", 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7709999999999998e-06 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230970+0000", 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5189999999999998e-06 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.230975+0000", 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.814e-06 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286220+0000", 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055245071 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286301+0000", 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1386999999999999e-05 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286404+0000", 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010257899999999999 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181728 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.474897533999993, 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055370672000000003, 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.789 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181728 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231235+0000", 2024-09-13T09:29:11.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231234+0000", 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231259+0000", 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4711000000000001e-05 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231268+0000", 2024-09-13T09:29:11.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9250000000000001e-06 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231276+0000", 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3410000000000003e-06 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286428+0000", 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055152435999999999 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286509+0000", 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1064999999999997e-05 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286606+0000", 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6236000000000005e-05 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:181730 3.18 3:1946ea69:::rbd_data.111827e6de71.0000000000000004:30 [read 2091008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:11.793 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 64.474713382999994, 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055384289000000003, 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 181730 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231419+0000", 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231418+0000", 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231424+0000", 2024-09-13T09:29:11.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3360000000000002e-06 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231436+0000", 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1907e-05 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.231444+0000", 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3289999999999994e-06 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:11.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286627+0000", 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055183216 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286698+0000", 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1222000000000004e-05 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.286803+0000", 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010517 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.797 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:11.798 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:11.798 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:11.799 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:11.799 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:11.795+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:11.900 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:11.897+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:11.912 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:11.912 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:11.912 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:11.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:12.001 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:11.997+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:12.030 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:12.030 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:12.030 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:12.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:12.104 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:12.099+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:12.159 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:12.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730707366999994, 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:12.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730692310999999, 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:12.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:12.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730682903000002, 2024-09-13T09:29:12.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:12.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730668609999995, 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:12.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:12.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730582904000002, 2024-09-13T09:29:12.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:12.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:12.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.183 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730226654999996, 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:12.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730179819, 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.192 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730140832999993, 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:12.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730120689000003, 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.197 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:12.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730101621000003, 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:12.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.206 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.730059531999999, 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:12.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729946029999994, 2024-09-13T09:29:12.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:12.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:12.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729927850999999, 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:12.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729918106, 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:12.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:12.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:12.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:12.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729907918999999, 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:12.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729815052999996, 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.230 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:12.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729711640999994, 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:12.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:12.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.239 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.729389763, 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:12.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:12.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:12.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.728973822, 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:12.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 65.728720112000005, 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:12.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:12.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.254 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:12.257 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:12.258 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:12.255+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:12.358 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:12.357+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:12.379 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:12.379 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:12.379 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:12.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:12.459 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:12.457+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:12.512 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:12.512 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:12.513 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:12.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:12.560 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:12.555+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:12.648 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.648 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 209.682993571, 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:12.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:12.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.653 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 146.56977382700001, 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.654 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:12.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 146.569375812, 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:12.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:12.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:12.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:12.664 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.894268936, 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:12.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:12.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.894254004, 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.671 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:12.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:12.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:12.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.677 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.89422089799999, 2024-09-13T09:29:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:12.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:12.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.894204605, 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:12.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:12.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.89418849, 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:12.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:12.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.89415744900001, 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:12.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.89412582599999, 2024-09-13T09:29:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:12.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:12.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.708 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.88117776499999, 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:12.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:12.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.881160709, 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:12.717 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.881144541, 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.880930879, 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:12.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:12.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.88091156799999, 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:12.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:12.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:12.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:12.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.88089517500001, 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:12.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:12.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.880879513, 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:12.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 145.880853159, 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:12.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.747 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 99.352340432000005, 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:12.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:12.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:12.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:12.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:12.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 94.240426317000001, 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:12.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:12.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:12.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:12.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:12.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:12.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:12.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:12.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:12.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:12.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:12.764 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:12.764 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:12.765 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:29:12.768 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:12.768 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:12.765+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:12.867 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:12.863+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:12.885 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:12.886 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:12.886 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:12.886 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:12.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:12.968 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:12.965+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:13.004 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:13.005 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:13.005 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:13.005 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:13.005 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:13.005 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:13.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:13.069 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:13.067+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:13.124 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 128.38110916599999, 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:13.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:13.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:13.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:13.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.138936415000003, 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:13.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.138913200000005, 2024-09-13T09:29:13.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:13.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:13.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:13.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.134784882999995, 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:13.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:13.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:13.147 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.134732322999994, 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:13.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.134695628000003, 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:13.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:13.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:13.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.156 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.134675998000006, 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:13.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:13.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.134653818999993, 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:13.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:13.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.130898036000005, 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:13.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:13.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.130872702000005, 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.170 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:13.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.130854865000003, 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:13.174 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:13.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:13.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.130842479999998, 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:13.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:13.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.130813719000002, 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:13.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.126661507999998, 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:13.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:13.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.126633220000002, 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:13.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.126607086000007, 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:13.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:13.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.126582448999997, 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:13.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:13.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.126499874999993, 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:13.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.126456972, 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.209 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.213 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 67.122831250000004, 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:13.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:13.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.217 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.218 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.218 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:13.218 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:13.218 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:13.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:13.220 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:13.220 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:13.220 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:13.220 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:13.220 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:13.215+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:13.234 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:13.257 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:13.278 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:13.282 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:13.283 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:13.328 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:13.327+0000 7fd695541700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 22204) UID: 0 2024-09-13T09:29:13.340 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:13.340 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:13.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:13.421 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:13.419+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:13.459 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:13.460 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:13.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:13.522 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:13.521+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:13.578 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:13.578 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:13.589 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:13.623 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:13.621+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:13.653 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0, 2, 4] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 6, 7, 3, 4] 2024-09-13T09:29:13.653 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:29:13.653 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 6, 7, 3, 4] 2024-09-13T09:29:13.653 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T09:29:13.653 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:29:13.668 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:29:13.712 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:13.712 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:13.723 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:13.724 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:13.723+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:13.825 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:13.821+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:13.831 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:13.831 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:13.841 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:13.926 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:13.925+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:13.954 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:13.965 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:14.027 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:14.023+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:14.070 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.108 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:14.128 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:14.125+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:14.229 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:14.227+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:14.285 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.285 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.294 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:14.330 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:14.327+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:14.457 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:14.455+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:14.532 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:14.531+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:14.542 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.542 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.553 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:14.633 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:14.631+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:14.658 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.658 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:14.734 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:14.733+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:14.780 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.780 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:14.835 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:14.833+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:14.935 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:14.933+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:14.953 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:14.954 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:14.964 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:15.036 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:15.033+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:15.070 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:15.070 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:15.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:15.138 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:15.135+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:15.237 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:15.238 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:15.235+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:15.284 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:15.339 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:15.337+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:15.352 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:15.353 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:15.363 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:15.421 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:15.440 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:15.437+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:15.443 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:15.462 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:15.470 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:15.470 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:15.470 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:15.480 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:15.483 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:15.509 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:15.532 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:15.533 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:15.537 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:15.537 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:15.580 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:15.579+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:15.590 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:15.590 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 130.84892319900001, 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:15.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:15.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.597 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.606750448, 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:15.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:15.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:15.604 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.606727233000001, 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.605 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:15.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.611 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.602598916000005, 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.612 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:15.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:15.616 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.602546356000005, 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.617 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.620 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:15.621 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.602509660999999, 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:15.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.625 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.626 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.602490031000002, 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.627 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.628 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.630 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.631 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.602467852000004, 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.632 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.633 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.634 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:15.635 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.636 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.637 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.598712069000001, 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.638 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:15.639 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.640 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.641 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:15.642 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.598686735000001, 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:15.643 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.644 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.645 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.646 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.598668898, 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.647 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.648 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.649 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.650 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:15.651 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.598656512999995, 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.652 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.653 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:15.654 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.655 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.656 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.598627751999999, 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.657 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.658 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.659 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.660 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.661 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.594475540999994, 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.662 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.663 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.664 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.665 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:15.666 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.594447252999998, 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.667 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.668 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:15.669 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.670 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:15.671 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.594421119000003, 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.672 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.673 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.674 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:15.675 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.676 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.594396481999993, 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.677 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.678 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:15.679 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.680 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.681 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.594313908000004, 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.682 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:15.683 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.684 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:15.685 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.594271004999996, 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:15.686 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:15.687 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.688 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.690 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.590645283000001, 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:15.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:15.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:15.693 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:15.695 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:15.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:29:15.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T09:29:15.697 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:15.693+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:15.698 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:15.798 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:15.795+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:15.816 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:15.816 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:15.816 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:15.816 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:15.826 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:15.899 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:15.896+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:15.933 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:15.933 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:15.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:16.000 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:15.999+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:16.054 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189765 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1406, mlcod=27'1)", 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3972658339999997, 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001809768, 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653436+0000", 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3040000000000005e-06 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653452+0000", 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6333000000000001e-05 2024-09-13T09:29:16.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653463+0000", 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0574000000000001e-05 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653574+0000", 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110859 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653617+0000", 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3325999999999998e-05 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655197+0000", 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015795729999999999 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655238+0000", 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1390999999999997e-05 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189786 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1409, mlcod=27'1)", 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3687559870000001, 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049715810000000001, 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681937+0000", 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681955+0000", 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7588000000000002e-05 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681979+0000", 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4073000000000001e-05 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681988+0000", 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2539999999999998e-06 2024-09-13T09:29:16.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682028+0000", 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0203000000000001e-05 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682044+0000", 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5586e-05 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686830+0000", 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047863819999999996 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686910+0000", 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9740999999999994e-05 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.064 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.9 193/191 [PushOp(3:951c3f22:::rbd_data.11189888a71f.0000000000000009:8, version: 32'36, data_included: [9458761~10034], data_size: 10034, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:951c3f22:::rbd_data.11189888a71f.0000000000000009:8@32'36, size: 9468795, copy_subset: [0~9468795], clone_subset: {}, snapset: 8=[]:{8=[8]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:9468795, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.711096+0000", 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3395985599999998, 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017354339999999999, 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711096+0000", 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711096+0000", 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711095+0000", 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711140+0000", 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5512000000000001e-05 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711171+0000", 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1263999999999997e-05 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711194+0000", 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2424000000000001e-05 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711348+0000", 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015456199999999999 2024-09-13T09:29:16.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.711377+0000", 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.889e-05 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.712831+0000", 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014539349999999999 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189812 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1413, mlcod=27'1)", 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3336546590000005, 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001885103, 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717038+0000", 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717050+0000", 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2282e-05 2024-09-13T09:29:16.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717070+0000", 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9337000000000001e-05 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717087+0000", 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983e-05 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717148+0000", 2024-09-13T09:29:16.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1765000000000002e-05 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717183+0000", 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4646e-05 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718857+0000", 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00167435 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718925+0000", 2024-09-13T09:29:16.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189821 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1415, mlcod=27'1)", 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3200786710000001, 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018352900000000001, 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730614+0000", 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730627+0000", 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2531000000000001e-05 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730652+0000", 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5663000000000001e-05 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730673+0000", 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0222000000000001e-05 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730779+0000", 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010659500000000001 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730822+0000", 2024-09-13T09:29:16.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3182000000000001e-05 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732358+0000", 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015359309999999999 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732451+0000", 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2554999999999997e-05 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189823 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1416, mlcod=27'1)", 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3165660240000001, 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017908609999999999, 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734127+0000", 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734140+0000", 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2886e-05 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734163+0000", 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3266999999999999e-05 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734179+0000", 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6552e-05 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734270+0000", 2024-09-13T09:29:16.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734315+0000", 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4555999999999998e-05 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735878+0000", 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015635079999999999 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735919+0000", 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0917999999999997e-05 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189827 3.1c e193/191 3:3831950c:::rbd_directory:head v 193'784, mlcod=30'142)", 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3072662230000001, 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001751297, 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743426+0000", 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743445+0000", 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8576e-05 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743504+0000", 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8888999999999998e-05 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743521+0000", 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743640+0000", 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119781 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743689+0000", 2024-09-13T09:29:16.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8134999999999999e-05 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745092+0000", 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014033419999999999 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745179+0000", 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7398000000000006e-05 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189844 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1417, mlcod=27'1)", 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2830577160000001, 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018671510000000001, 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767635+0000", 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767648+0000", 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3448000000000001e-05 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767681+0000", 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2508000000000001e-05 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767705+0000", 2024-09-13T09:29:16.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4511e-05 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767771+0000", 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5772999999999995e-05 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767825+0000", 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3628999999999998e-05 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769445+0000", 2024-09-13T09:29:16.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016198460000000001 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769504+0000", 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9043999999999999e-05 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189857 3.14 e193/191 3:283508b2:::rbd_data.1118fb4e889d.0000000000000002:head v 193'607, mlcod=0'0)", 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2666923780000001, 2024-09-13T09:29:16.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030577590000000002, 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784000+0000", 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784015+0000", 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4663e-05 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784043+0000", 2024-09-13T09:29:16.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8521999999999999e-05 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784056+0000", 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2591e-05 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784142+0000", 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6336999999999997e-05 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784196+0000", 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4323999999999997e-05 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.786969+0000", 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027724210000000002 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787060+0000", 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0841999999999995e-05 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189863 3.11 e193/191 3:88e7d49d:::rbd_data.1118fb4e889d.0000000000000008:head v 193'877, mlcod=0'0)", 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.785515+0000", 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2651796510000004, 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001700072, 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785515+0000", 2024-09-13T09:29:16.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785515+0000", 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785513+0000", 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785526+0000", 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2907e-05 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785555+0000", 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9281999999999998e-05 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785569+0000", 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3373999999999999e-05 2024-09-13T09:29:16.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785657+0000", 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8170000000000005e-05 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785699+0000", 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2410000000000002e-05 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787154+0000", 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00145426 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787215+0000", 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1075999999999996e-05 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189862 3.13 e193/191 3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head v 193'1112, mlcod=26'1)", 2024-09-13T09:29:16.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2650993289999999, 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00238909, 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785594+0000", 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785603+0000", 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785615+0000", 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2490999999999999e-05 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785623+0000", 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785736+0000", 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011302999999999999 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785788+0000", 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1433e-05 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787873+0000", 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020855499999999998 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787984+0000", 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110661 2024-09-13T09:29:16.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189865 3.1a e193/191 3:5cf1de86:::rbd_data.1118fb4e889d.000000000000000a:head v 193'674, mlcod=189'672)", 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.787376+0000", 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2633179219999997, 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00174953, 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787376+0000", 2024-09-13T09:29:16.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787376+0000", 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787374+0000", 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787389+0000", 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4419999999999999e-05 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787412+0000", 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3788e-05 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787425+0000", 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2198000000000001e-05 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787566+0000", 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014166600000000001 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.788106+0000", 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053969 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.789054+0000", 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094773399999999999 2024-09-13T09:29:16.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.789126+0000", 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2084999999999998e-05 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189944 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.1511917829999998, 2024-09-13T09:29:16.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.77023277800000001, 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189944 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899501+0000", 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899511+0000", 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9010000000000006e-06 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899528+0000", 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7445999999999999e-05 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899545+0000", 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899618+0000", 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2534999999999996e-05 2024-09-13T09:29:16.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899627+0000", 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.667960+0000", 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.76833267900000002 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.668076+0000", 2024-09-13T09:29:16.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.669735+0000", 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587539999999999 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.3 193/191 [PushOp(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6, version: 30'16, data_included: [1126604~6238], data_size: 6238, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6@30'16, size: 10485760, copy_subset: [0~10485760], clone_subset: {}, snapset: 6=[]:{6=[6]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:10485760, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.9401500860000001, 2024-09-13T09:29:16.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017837389999999999, 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110543+0000", 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110579+0000", 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5716000000000002e-05 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110617+0000", 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8492999999999999e-05 2024-09-13T09:29:16.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110646+0000", 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9224999999999999e-05 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110840+0000", 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019405299999999999 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110862+0000", 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2195999999999999e-05 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.112328+0000", 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014654100000000001 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.118 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.3 193/191 [PushOp(3:c6863065:::rbd_data.11189888a71f.0000000000000008:8, version: 32'36, data_included: [1070391~31498,2842054~60183], data_size: 91681, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:c6863065:::rbd_data.11189888a71f.0000000000000008:8@32'36, size: 2902237, copy_subset: [0~2902237], clone_subset: {}, snapset: 8=[]:{8=[8]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2902237, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.125106+0000", 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.9255886709999999, 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017079910000000001, 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125106+0000", 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125106+0000", 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125104+0000", 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125340+0000", 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00023545799999999999 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125369+0000", 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9649999999999999e-05 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125410+0000", 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.066e-05 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125505+0000", 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5203000000000003e-05 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.125521+0000", 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5948e-05 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.126814+0000", 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012923780000000001 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e193,191 3:2c83699f:::rbd_data.111817fd4ca8.000000000000011a:head,68'129)", 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3904994749999999, 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020817029999999999, 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660193+0000", 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660204+0000", 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660225+0000", 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0594999999999998e-05 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660242+0000", 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-05 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662101+0000", 2024-09-13T09:29:16.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018583860000000001 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662117+0000", 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5886999999999999e-05 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662276+0000", 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159901 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e194,191 3:2dffdd96:::rbd_data.111817fd4ca8.0000000000000162:head,68'133)", 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3858862840000001, 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021107690000000002, 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664807+0000", 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664814+0000", 2024-09-13T09:29:16.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664829+0000", 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5130000000000001e-05 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664844+0000", 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4782e-05 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665589+0000", 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074510899999999998 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665598+0000", 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9199999999999993e-06 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666919+0000", 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00132099 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(osd.7.0:5052 3.f e194/193 3:f5427e30:::rbd_data.11189fb5ddd6.0000000000000003:b v 194'480, mlcod=0'0)", 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3842530449999999, 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060970809999999999, 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666447+0000", 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666455+0000", 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666461+0000", 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666496+0000", 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5063e-05 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666515+0000", 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672509+0000", 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059937419999999998 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672538+0000", 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9645000000000002e-05 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189946 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.3776574149999998, 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068095500000000002, 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:16.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189946 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673035+0000", 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673044+0000", 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673060+0000", 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5872999999999999e-05 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673068+0000", 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1759999999999993e-06 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673112+0000", 2024-09-13T09:29:16.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3998000000000001e-05 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673189+0000", 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6330999999999998e-05 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673305+0000", 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011642 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.678066+0000", 2024-09-13T09:29:16.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047612519999999997 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679714+0000", 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001647665 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679755+0000", 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0941e-05 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679846+0000", 2024-09-13T09:29:16.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1558000000000007e-05 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.1c 194/191 [PushOp(3:3985481d:::rbd_data.111851704b2b.0000000000000016:9, version: 34'163, data_included: [2466136~7078], data_size: 7078, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:3985481d:::rbd_data.111851704b2b.0000000000000016:9@34'163, size: 2473214, copy_subset: [0~2473214], clone_subset: {}, snapset: 9=[]:{9=[9]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:2473214, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:13.174740+0000", 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.8759538330000001, 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016729099999999999, 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:16.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174740+0000", 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174740+0000", 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174739+0000", 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:16.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174776+0000", 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6806e-05 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174815+0000", 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9100000000000002e-05 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174845+0000", 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9895000000000001e-05 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:16.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.174999+0000", 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015425700000000001 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.175021+0000", 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2476e-05 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:13.176413+0000", 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013919959999999999 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:16.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:16.145 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:16.146 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:16.146 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:16.143+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:16.196 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:29:16.247 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:16.244+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:16.278 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:16.279 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:16.279 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:16.279 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:16.289 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:16.347 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:16.343+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:16.391 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:16.391 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:16.391 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:16.401 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:16.448 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:16.447+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:16.509 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:16.509 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 154.94332396799999, 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.510 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:16.511 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:16.512 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.513 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:16.514 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.515 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 122.355665589, 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.516 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:16.517 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:16.518 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.519 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.520 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 70.225373464, 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.521 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.522 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:16.523 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.524 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.641163012999996, 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.525 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.526 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:16.527 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.528 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:16.529 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239829381000007, 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.530 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:16.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:16.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239811372999995, 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:16.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:16.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239794033999999, 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.540 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:16.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239776777000003, 2024-09-13T09:29:16.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:16.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239759241000002, 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:16.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:16.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:16.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239742171000003, 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:16.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:16.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239728511999999, 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:16.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239718839000005, 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:16.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:16.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:16.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239704294000006, 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:16.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:16.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239687626999995, 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:16.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:16.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:16.577 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.239670536000006, 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.578 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.237814959999994, 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:16.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.237798851999997, 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:16.587 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:16.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:16.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.237784488000003, 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.592 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:16.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.596 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.237770659999995, 2024-09-13T09:29:16.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:16.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:16.602 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 69.237752959999995, 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:16.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.609 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:16.611 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:16.611 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:16.608+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:16.712 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:16.708+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:16.729 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:16.729 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:16.730 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:16.730 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:16.740 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:16.813 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:16.811+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:16.852 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:16.853 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:16.853 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:16.853 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:16.853 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:16.863 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:16.913 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:16.911+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:16.972 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:16.972 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393533157999997, 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.973 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:16.974 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:16.975 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.976 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.977 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393520921000004, 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.978 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393505524000005, 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.982 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:16.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393490649, 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:16.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.991 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393473439999994, 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:16.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.996 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393452984999996, 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.997 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:16.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:16.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.001 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393434905000007, 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:17.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:17.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393418529000002, 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:17.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393403178, 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:17.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.015 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393386473999996, 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:17.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:17.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:17.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:17.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393368391999999, 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.025 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.026 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393351647000003, 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:17.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:17.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.393306988000006, 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.031 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:17.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:17.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:17.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:17.036 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.390357547999997, 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:17.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:17.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.390337009999996, 2024-09-13T09:29:17.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:17.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:17.045 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.390314395000004, 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.046 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:17.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.390295527000006, 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:17.050 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:17.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.054 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.390260932999993, 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:17.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.058 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.385982028000001, 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:17.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:17.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:17.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:17.065 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 89.385918055999994, 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.066 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:17.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:17.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:17.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:17.073 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:17.073 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:17.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:17.075 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:17.072+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:17.175 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:17.172+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:17.191 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:17.192 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:17.192 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:17.192 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:17.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:17.276 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:17.272+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:17.305 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:17.305 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:17.305 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:17.305 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:17.305 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:17.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:17.377 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:17.375+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:17.425 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:17.425 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.997407224, 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.997392168000005, 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:17.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:17.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.997382759999994, 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.997368467000001, 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:17.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:17.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.997282760999994, 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:17.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:17.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996926512000002, 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:17.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996879676000006, 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:17.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:17.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996840689999999, 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:17.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996820545999995, 2024-09-13T09:29:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:17.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996801477999995, 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:17.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996759389000005, 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:17.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:17.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:17.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996645887, 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:17.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996627708000005, 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:17.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:17.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.484 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996617963000006, 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:17.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996607776000005, 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:17.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996514910000002, 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996411498000001, 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:17.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:17.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:17.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.996089620000006, 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:17.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.995673679000006, 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.511 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.995419968999997, 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:17.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:17.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:17.516 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:17.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:17.518 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:17.516+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:17.538 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:17.618 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:17.615+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:17.678 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:17.678 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:17.679 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:17.679 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:17.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:17.699 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:17.719 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:17.720 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:17.716+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:17.741 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:17.765 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:17.790 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:17.793 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:17.793 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:17.793 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:17.793 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:17.793 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:17.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:17.813 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:17.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:17.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:17.813 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:17.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:17.813 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:17.814 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:17.814 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:17.814 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:17.818 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:17.818 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:17.845 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:17.844+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:17.921 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:17.919+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:17.957 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:17.957 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:17.967 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:18.023 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:18.019+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.082 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.092 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:18.124 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:18.120+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:18.208 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.209 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:29:18.219 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:18.224 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:18.220+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:18.325 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:18.323+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:18.378 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.378 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.388 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:18.426 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:18.423+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:18.504 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.504 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:18.527 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:18.523+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:18.617 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.617 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:18.629 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:18.624+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:18.729 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:18.728+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:18.734 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.734 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.744 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:18.830 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:18.827+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:18.862 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.862 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.872 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:18.931 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:18.927+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:18.984 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:18.984 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:18.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:19.032 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:19.028+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:19.133 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:19.131+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:19.147 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.147 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.157 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:19.208 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:29:19.234 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:19.231+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:19.270 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.271 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:19.335 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:19.332+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:19.436 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:19.432+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:19.442 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.442 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.453 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:19.537 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:19.536+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:19.560 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.560 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:19.638 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:19.636+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:19.689 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.690 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:19.739 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:19.736+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:19.753 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T09:29:19.754 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-13T09:29:19.754 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:29:19.754 INFO:tasks.thrashosds.thrasher:remote for exp osd 5 is ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:29:19.754 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-13T09:29:19.760 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:29:19.760 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-13T09:29:19.805 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.805 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:19.820 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:19.840 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:19.839+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:19.928 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:19.928 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:19.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:19.941 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:19.936+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:19.977 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:20.000 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:20.019 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:20.038 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:20.039 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:20.039 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:20.041 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:20.040+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:20.049 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:20.062 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:20.088 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:20.092 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:20.093 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:20.142 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:20.140+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:20.155 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:20.156 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:20.156 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:20.156 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:20.156 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:20.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:20.243 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:20.240+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:20.321 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:20.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 217.35562346200001, 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:20.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:20.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 154.24240371799999, 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:20.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:20.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 154.24200570299999, 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:20.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:20.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.56689882699999, 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:20.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:20.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:20.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:20.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:20.344 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.56688389499999, 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:20.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.566850789, 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:20.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:20.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.56683449600001, 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:20.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:20.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.360 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.56681838099999, 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.361 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:20.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.56678733999999, 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:20.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.56675571700001, 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:20.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:20.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.553807656, 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:20.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.55379060000001, 2024-09-13T09:29:20.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:20.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.55377443200001, 2024-09-13T09:29:20.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:20.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.55356076999999, 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:20.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.553541459, 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:20.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.403 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.55352506599999, 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:20.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:20.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.55350940400001, 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.409 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:20.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:20.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 153.55348305000001, 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.414 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:20.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:20.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.02497032300001, 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:20.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:20.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:20.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:20.427 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.913056208, 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:20.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:20.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:20.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.433 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:20.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:20.434 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:20.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:20.436 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:20.431+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:20.537 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:20.535+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:20.552 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:20.552 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:20.552 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:20.552 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:20.562 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:20.638 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:20.636+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:20.716 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:20.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:20.716 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:20.716 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:20.716 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:20.726 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:20.739 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:20.736+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:20.833 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 136.092441017, 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:20.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.850268266, 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.850245051000002, 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:20.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.846116734000006, 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.850 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:20.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.846064174000006, 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.854 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:20.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:20.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.858 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.846027479, 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:20.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:20.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:20.863 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.846007849000003, 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.865 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:20.866 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:20.867 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.845985670000005, 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.868 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.870 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.871 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842229887000002, 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.872 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.873 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.874 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.875 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842204553000002, 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.876 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:20.877 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.878 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.879 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.880 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842186716000001, 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.881 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.882 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.883 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.884 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.842174330999995, 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.885 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.886 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.887 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.888 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.84214557, 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.889 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.890 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:20.891 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.892 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:20.893 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.837993358999995, 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.894 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.895 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.896 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.897 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.837965070999999, 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:20.898 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:20.899 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:20.900 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:20.901 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.837938937000004, 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.902 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.903 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:20.904 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.905 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.906 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.837914299999994, 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:20.907 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.908 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.909 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.910 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.837831726000005, 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.911 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:20.912 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:20.913 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:20.914 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.916 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.837788822999997, 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:20.917 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:20.918 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:20.919 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.920 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.834163101000001, 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:20.921 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.922 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:20.923 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:20.924 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.925 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.926 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:20.926 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:20.926 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:20.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:20.928 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:20.924+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.d 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.c 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.2 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.1f 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.1c 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.5 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.1d 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.18 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.17 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:3.16 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:1.4 2024-09-13T09:29:20.944 INFO:teuthology.orchestra.run.smithi099.stdout:1.1 2024-09-13T09:29:21.028 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:21.027+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:21.031 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:21.031 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:21.032 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:21.032 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:21.041 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:21.129 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:21.128+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:21.196 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:21.196 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:21.197 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:21.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:21.222 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:29:21.230 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:21.228+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:21.312 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:21.312 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:21.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:21.312 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:21.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189765 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1406, mlcod=27'1)", 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6550694910000008, 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001809768, 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653436+0000", 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3040000000000005e-06 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653452+0000", 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6333000000000001e-05 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653463+0000", 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0574000000000001e-05 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653574+0000", 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110859 2024-09-13T09:29:21.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653617+0000", 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3325999999999998e-05 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655197+0000", 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015795729999999999 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655238+0000", 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1390999999999997e-05 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189786 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1409, mlcod=27'1)", 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6265596440000003, 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049715810000000001, 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.317 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681937+0000", 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681955+0000", 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7588000000000002e-05 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681979+0000", 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4073000000000001e-05 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681988+0000", 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2539999999999998e-06 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682028+0000", 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0203000000000001e-05 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682044+0000", 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5586e-05 2024-09-13T09:29:21.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686830+0000", 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047863819999999996 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686910+0000", 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9740999999999994e-05 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189812 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1413, mlcod=27'1)", 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5914583160000007, 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001885103, 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717038+0000", 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717050+0000", 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2282e-05 2024-09-13T09:29:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717070+0000", 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9337000000000001e-05 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717087+0000", 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983e-05 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717148+0000", 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1765000000000002e-05 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717183+0000", 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4646e-05 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718857+0000", 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00167435 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718925+0000", 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189821 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1415, mlcod=27'1)", 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5778823279999994, 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018352900000000001, 2024-09-13T09:29:21.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730614+0000", 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730627+0000", 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2531000000000001e-05 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730652+0000", 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5663000000000001e-05 2024-09-13T09:29:21.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730673+0000", 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0222000000000001e-05 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730779+0000", 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010659500000000001 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730822+0000", 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3182000000000001e-05 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732358+0000", 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015359309999999999 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732451+0000", 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2554999999999997e-05 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189823 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1416, mlcod=27'1)", 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5743696810000003, 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017908609999999999, 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734127+0000", 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734140+0000", 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2886e-05 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734163+0000", 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3266999999999999e-05 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734179+0000", 2024-09-13T09:29:21.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6552e-05 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734270+0000", 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734315+0000", 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4555999999999998e-05 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735878+0000", 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015635079999999999 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735919+0000", 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0917999999999997e-05 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189827 3.1c e193/191 3:3831950c:::rbd_directory:head v 193'784, mlcod=30'142)", 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5650698799999994, 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001751297, 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743426+0000", 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743445+0000", 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8576e-05 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743504+0000", 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8888999999999998e-05 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743521+0000", 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743640+0000", 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119781 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743689+0000", 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8134999999999999e-05 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745092+0000", 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014033419999999999 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745179+0000", 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7398000000000006e-05 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189844 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1417, mlcod=27'1)", 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:21.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5408613730000003, 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018671510000000001, 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:21.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767635+0000", 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767648+0000", 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3448000000000001e-05 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767681+0000", 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2508000000000001e-05 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767705+0000", 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4511e-05 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767771+0000", 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5772999999999995e-05 2024-09-13T09:29:21.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767825+0000", 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3628999999999998e-05 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769445+0000", 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016198460000000001 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769504+0000", 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9043999999999999e-05 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189857 3.14 e193/191 3:283508b2:::rbd_data.1118fb4e889d.0000000000000002:head v 193'607, mlcod=0'0)", 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5244960350000003, 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030577590000000002, 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.345 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784000+0000", 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784015+0000", 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4663e-05 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784043+0000", 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8521999999999999e-05 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784056+0000", 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2591e-05 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784142+0000", 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6336999999999997e-05 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784196+0000", 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4323999999999997e-05 2024-09-13T09:29:21.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.786969+0000", 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027724210000000002 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787060+0000", 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0841999999999995e-05 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189862 3.13 e193/191 3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head v 193'1112, mlcod=26'1)", 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5229029860000001, 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00238909, 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785594+0000", 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785603+0000", 2024-09-13T09:29:21.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785615+0000", 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2490999999999999e-05 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785623+0000", 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785736+0000", 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011302999999999999 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785788+0000", 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1433e-05 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787873+0000", 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020855499999999998 2024-09-13T09:29:21.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787984+0000", 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110661 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189944 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.40899544, 2024-09-13T09:29:21.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.77023277800000001, 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189944 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899501+0000", 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899511+0000", 2024-09-13T09:29:21.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9010000000000006e-06 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899528+0000", 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7445999999999999e-05 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899545+0000", 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899618+0000", 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2534999999999996e-05 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899627+0000", 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.667960+0000", 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.76833267900000002 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.668076+0000", 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.669735+0000", 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587539999999999 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.3 193/191 [PushOp(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6, version: 30'16, data_included: [1126604~6238], data_size: 6238, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6@30'16, size: 10485760, copy_subset: [0~10485760], clone_subset: {}, snapset: 6=[]:{6=[6]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:10485760, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.1979537429999993, 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017837389999999999, 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110543+0000", 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110579+0000", 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5716000000000002e-05 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110617+0000", 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8492999999999999e-05 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110646+0000", 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9224999999999999e-05 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110840+0000", 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019405299999999999 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110862+0000", 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2195999999999999e-05 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.112328+0000", 2024-09-13T09:29:21.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014654100000000001 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e193,191 3:2c83699f:::rbd_data.111817fd4ca8.000000000000011a:head,68'129)", 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.6483031320000006, 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020817029999999999, 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660193+0000", 2024-09-13T09:29:21.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660204+0000", 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660225+0000", 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0594999999999998e-05 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660242+0000", 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-05 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662101+0000", 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018583860000000001 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662117+0000", 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5886999999999999e-05 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662276+0000", 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159901 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e194,191 3:2dffdd96:::rbd_data.111817fd4ca8.0000000000000162:head,68'133)", 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.6436899409999999, 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021107690000000002, 2024-09-13T09:29:21.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664807+0000", 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664814+0000", 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664829+0000", 2024-09-13T09:29:21.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5130000000000001e-05 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664844+0000", 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4782e-05 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665589+0000", 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074510899999999998 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665598+0000", 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9199999999999993e-06 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666919+0000", 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00132099 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(osd.7.0:5052 3.f e194/193 3:f5427e30:::rbd_data.11189fb5ddd6.0000000000000003:b v 194'480, mlcod=0'0)", 2024-09-13T09:29:21.372 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.6420567019999996, 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060970809999999999, 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666447+0000", 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666455+0000", 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666461+0000", 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666496+0000", 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5063e-05 2024-09-13T09:29:21.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666515+0000", 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672509+0000", 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059937419999999998 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672538+0000", 2024-09-13T09:29:21.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9645000000000002e-05 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189946 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.635461072, 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068095500000000002, 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:21.377 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189946 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673035+0000", 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673044+0000", 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673060+0000", 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5872999999999999e-05 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673068+0000", 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1759999999999993e-06 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673112+0000", 2024-09-13T09:29:21.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3998000000000001e-05 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673189+0000", 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6330999999999998e-05 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673305+0000", 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011642 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.678066+0000", 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047612519999999997 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679714+0000", 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001647665 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679755+0000", 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0941e-05 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679846+0000", 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1558000000000007e-05 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190167 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0347736660000004, 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024116239999999998, 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190167 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273722+0000", 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273736+0000", 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.345e-05 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273767+0000", 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1359000000000002e-05 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273783+0000", 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5549999999999999e-05 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273893+0000", 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274045+0000", 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015171299999999999 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274391+0000", 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034663899999999999 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.275713+0000", 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013217630000000001 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276007+0000", 2024-09-13T09:29:21.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029441100000000001 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276074+0000", 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276136+0000", 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1420999999999994e-05 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190169 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.030387084, 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029159210000000001, 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190169 2024-09-13T09:29:21.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278109+0000", 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278124+0000", 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5449000000000001e-05 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278157+0000", 2024-09-13T09:29:21.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2602999999999999e-05 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278178+0000", 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1195999999999998e-05 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278298+0000", 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278456+0000", 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015802899999999999 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278817+0000", 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036107899999999999 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280193+0000", 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001375392 2024-09-13T09:29:21.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280781+0000", 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058817699999999995 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280865+0000", 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3697999999999997e-05 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281027+0000", 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016203300000000001 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190170 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0273692920000004, 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023661509999999999, 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190170 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:21.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281127+0000", 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281140+0000", 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:29:21.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281164+0000", 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4369999999999999e-05 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281182+0000", 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8016000000000002e-05 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281324+0000", 2024-09-13T09:29:21.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014187999999999999 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281446+0000", 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121198 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281690+0000", 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024418399999999999 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283024+0000", 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013344030000000001 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283315+0000", 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029132399999999998 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283392+0000", 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6452000000000006e-05 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283495+0000", 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010284400000000001 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190173 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0234598760000004, 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025929310000000001, 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190173 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:21.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285036+0000", 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285047+0000", 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1083e-05 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285070+0000", 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3050000000000001e-05 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285082+0000", 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1749999999999999e-05 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285169+0000", 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6514000000000006e-05 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285332+0000", 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016333 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285618+0000", 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028579000000000001 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.286946+0000", 2024-09-13T09:29:21.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001327679 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287517+0000", 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00057192199999999999 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287571+0000", 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3903000000000002e-05 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287631+0000", 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.961e-05 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:190198 3.11 e198/197 3:8daa9abf:::rbd_data.11189888a71f.0000000000000001:head v 198'880, mlcod=193'877)", 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9911370509999999, 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023753540000000001, 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:21.409 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:21.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317359+0000", 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317375+0000", 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6348999999999999e-05 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317416+0000", 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0562000000000002e-05 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317432+0000", 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5957e-05 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317658+0000", 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000225768 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317716+0000", 2024-09-13T09:29:21.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7819000000000003e-05 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319644+0000", 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019285999999999999 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319736+0000", 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2034999999999996e-05 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:21.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:21.414 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:21.415 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:21.412+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:21.415 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:21.516 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:21.512+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:21.526 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:21.526 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:21.526 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:21.526 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:21.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.17 --file /var/log/ceph/exp.3.17.5 2024-09-13T09:29:21.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:21.617 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:21.616+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:21.648 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:21.648 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:21.649 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:21.649 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:21.649 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:21.658 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:21.718 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:21.716+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:21.770 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:21.770 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:21.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:21.770 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 160.205618638, 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:21.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:21.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 127.617960259, 2024-09-13T09:29:21.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:21.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:21.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 75.487668134000003, 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.903457682999999, 2024-09-13T09:29:21.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:21.790 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502124050999996, 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:21.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:21.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502106042999998, 2024-09-13T09:29:21.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:21.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.799 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502088704000002, 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:21.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:21.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502071447000006, 2024-09-13T09:29:21.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:21.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:21.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:21.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502053911000004, 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:21.809 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:21.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502036841000006, 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:21.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:21.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.818 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502023182000002, 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:21.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.502013508999994, 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:21.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.827 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.501998963999995, 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:21.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:21.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:21.832 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.501982296999998, 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:21.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:21.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.501965205999994, 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:21.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.841 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.500109629999997, 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.500093522, 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:21.850 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.500079158000005, 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:21.851 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.853 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.854 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.855 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.500065329999998, 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.856 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.857 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.858 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:21.859 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 74.500047629999997, 2024-09-13T09:29:21.860 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.861 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:21.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:21.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:21.864 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:21.866 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:21.867 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:21.863+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:21.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T09:29:21.966 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:21.963+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:22.023 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:22.023 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:22.023 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:22.023 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:22.033 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:22.067 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:22.063+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:22.094 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:22.144 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:22.144 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:22.145 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:22.145 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:22.153 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:22.168 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:22.164+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:22.203 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:29:22.238 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:22.258 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:22.270 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:22.267+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:22.281 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:22.303 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:22.304 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:22.304 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:22.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:22.327 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:22.353 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:22.353 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:22.353 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:22.353 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:22.354 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:22.356 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:22.357 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:22.371 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:22.367+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:22.432 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:22.432 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:22.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:22.472 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:22.468+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:22.546 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:22.546 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:22.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:22.573 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:22.568+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:22.660 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:22.660 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:22.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:22.714 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:22.712+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:Exporting 3.17 info 3.17( v 177'761 (0'0,177'761] local-lis/les=60/61 n=9 ec=21/21 lis/c=60/60 les/c/f=61/61/0 sis=60) 2024-09-13T09:29:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:e89855de:::rbd_object_map.1118e98b86fc:head# 2024-09-13T09:29:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:eaa625ad:::rbd_data.111867511a81.0000000000000010:head# 2024-09-13T09:29:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:eb7ede79:::rbd_data.1118a28cd5c4.00000000000000ca:head# 2024-09-13T09:29:22.741 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ec7c1922:::rbd_data.1118a28cd5c4.000000000000008b:head# 2024-09-13T09:29:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ed4f9a1c:::rbd_data.111888f95246.000000000000000f:head# 2024-09-13T09:29:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ed4f9ac5:::rbd_data.1118a28cd5c4.000000000000002d:head# 2024-09-13T09:29:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ed7fd31c:::rbd_object_map.11185454a1c8:head# 2024-09-13T09:29:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ee110ec6:::rbd_data.111867511a81.0000000000000003:5# 2024-09-13T09:29:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:Read #3:ee110ec6:::rbd_data.111867511a81.0000000000000003:head# 2024-09-13T09:29:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:Export successful 2024-09-13T09:29:22.775 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:22.772+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:22.781 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:22.781 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:22.791 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:22.875 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:22.872+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:22.892 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:22.892 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:22.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:22.977 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:22.974+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:23.051 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.051 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:23.077 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:23.076+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:23.179 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.179 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.179 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:23.174+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:23.189 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:23.280 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:23.276+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:23.300 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.300 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.310 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:23.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.17 2024-09-13T09:29:23.380 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:23.378+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:23.415 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.415 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:23.481 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:23.478+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:23.531 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.531 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.541 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:23.582 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:23.578+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:23.644 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.644 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:23.683 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:23.678+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:23.766 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.766 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.777 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:23.821 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:23.818+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:23.885 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:23.884+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:23.898 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:23.899 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:23.910 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:23.987 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:23.984+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:24.014 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:24.014 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:24.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:24.087 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:24.082+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:24.129 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:24.129 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:24.139 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:24.188 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:24.182+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:24.218 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:29:24.289 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:24.284+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:24.295 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:24.295 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:24.306 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:24.358 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:24.390 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:24.388+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:24.410 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:24.410 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:24.420 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:24.490 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:24.486+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:24.493 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:24.513 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950791155000005, 2024-09-13T09:29:24.530 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.531 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.532 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.533 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:24.534 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950778917999997, 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.535 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.536 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:24.537 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.538 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.539 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950763520999999, 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.540 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.541 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:24.542 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.543 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950748645999994, 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.544 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.545 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.546 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:24.548 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950731437000002, 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.549 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:24.550 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.551 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:24.552 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.553 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950710982000004, 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.554 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.555 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:24.556 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.557 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950692902, 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.558 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.559 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:24.560 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:24.561 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.562 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950676525999995, 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.563 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:24.564 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.565 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.566 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950661174999993, 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.567 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.568 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.569 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:24.570 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.571 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950644471000004, 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.572 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:24.573 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.574 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:24.575 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950626389000007, 2024-09-13T09:29:24.576 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:24.577 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.578 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:24.579 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.580 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950609643999996, 2024-09-13T09:29:24.581 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.582 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:24.583 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.584 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.585 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.950564985, 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:24.586 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.587 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.589 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.590 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.947615545000005, 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.591 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.592 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.593 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:24.594 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.595 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.947595007000004, 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.596 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.597 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:24.598 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.599 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.600 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.947572391999998, 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.601 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.602 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.603 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:24.604 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.947553524, 2024-09-13T09:29:24.605 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.606 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:24.607 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.608 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.609 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.947518930000001, 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.610 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.611 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.612 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.613 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.614 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.943240024999994, 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.615 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.616 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:24.617 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.618 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.619 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 96.943176053000002, 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.620 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:24.621 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.622 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:24.623 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:24.624 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:24.625 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:24.626 INFO:teuthology.orchestra.run.smithi099.stdout: marking collection for removal 2024-09-13T09:29:24.626 INFO:teuthology.orchestra.run.smithi099.stdout:setting '_remove' omap key 2024-09-13T09:29:24.626 INFO:teuthology.orchestra.run.smithi099.stdout:finish_remove_pgs 3.17_head removing 3.17 2024-09-13T09:29:24.626 INFO:teuthology.orchestra.run.smithi099.stdout:Remove successful 2024-09-13T09:29:24.626 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:24.624+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:24.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:24.628 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:24.628 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:24.628 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:24.628 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:24.629 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:24.630 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:24.630 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:24.726 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:24.722+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:24.747 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:24.747 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:24.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:24.827 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:24.822+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:24.864 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:24.864 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:24.864 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:24.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:24.927 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:24.924+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:24.981 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:24.981 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:24.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.553155343, 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:24.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:24.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:24.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.553140287000005, 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:24.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:24.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:24.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.553130878999994, 2024-09-13T09:29:24.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:24.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:24.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.553116586000002, 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:24.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:24.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:24.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:24.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.553030879999994, 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:25.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:25.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552674631000002, 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:25.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552627795000006, 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:25.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552588809, 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.014 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:25.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:25.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:25.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:25.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552568664999995, 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552549596999995, 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552507508000005, 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:25.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:25.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552394006, 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:25.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:25.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552375827000006, 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:25.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:25.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552366082000006, 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:25.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552355895000005, 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:25.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:25.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552263029000002, 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:25.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:25.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:25.056 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.552159617000001, 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:25.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.551837739000007, 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.061 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:25.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.551421798000007, 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:25.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:25.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.551168087999997, 2024-09-13T09:29:25.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:25.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.075 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:25.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:25.077 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:25.074+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:25.095 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-13T09:29:25.177 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:25.174+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:25.251 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:25.251 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:25.251 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:25.251 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:25.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:25.278 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:25.274+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:25.369 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:25.369 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:25.379 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:25.379 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:25.376+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:25.480 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:25.476+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:25.491 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 222.52729737199999, 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:25.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:25.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:25.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 159.414077628, 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:25.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:25.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 159.413679613, 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:25.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:25.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:25.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:25.506 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.738572737, 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:25.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:25.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.738557805, 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:25.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.73852469900001, 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:25.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.73850840599999, 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.520 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:25.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:25.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.738492291, 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.525 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:25.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.73846125, 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.529 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:25.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.73842962699999, 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.534 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:25.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.538 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72548156600001, 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:25.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:25.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72546450999999, 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.543 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.547 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72544834199999, 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:25.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:25.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72523468, 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:25.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72521536900001, 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:25.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.725198976, 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:25.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72518331399999, 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.566 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:25.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:25.570 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 158.72515695999999, 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:25.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:25.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.196644233, 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:25.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:25.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.582 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.084730118, 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:25.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:25.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:25.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:25.588 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:29:25.590 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:25.590 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:25.586+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:25.689 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:25.686+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:25.748 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:25.748 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:25.748 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:25.748 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:25.758 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:25.791 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:25.788+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:25.860 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:25.860 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:25.860 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:25.860 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:25.860 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:25.869 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:25.891 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:25.888+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:25.978 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 141.238436404, 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:25.979 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.980 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.981 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:25.982 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:25.983 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.984 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:25.985 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.996263653, 2024-09-13T09:29:25.986 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.987 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:25.988 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.989 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:25.990 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.996240438000001, 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:25.991 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:25.992 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.993 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:25.994 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:25.995 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.992112121000005, 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:25.996 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:25.997 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.998 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:25.999 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:26.000 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:26.001 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.992059561000005, 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:26.002 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.003 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.004 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.005 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.992022865999999, 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:26.006 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.007 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.008 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.009 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.010 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.992003236000002, 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.011 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:26.012 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.013 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.014 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.015 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.991981057000004, 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.016 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:26.017 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:26.018 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.019 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.988225274000001, 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:26.020 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.021 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:26.022 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.023 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.024 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.988199940000001, 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.025 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.026 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:26.027 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.028 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.029 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.988182103, 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.030 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:26.031 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.032 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:26.033 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.034 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.988169717999995, 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.035 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.036 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:26.037 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.038 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.988140956999999, 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.039 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:26.040 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.041 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.042 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.043 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.983988745999994, 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.044 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.045 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:26.046 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.047 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.983960457999999, 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.048 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:26.049 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:26.050 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.051 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.052 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.053 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.983934324000003, 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:26.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:26.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.983909686999993, 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.058 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:26.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.062 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.983827113000004, 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:26.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:26.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.983784209999996, 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:26.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:26.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.071 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 79.980158488000001, 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:26.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:26.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:26.076 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:26.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:26.078 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:26.074+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:26.179 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:26.176+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:26.194 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:26.194 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:26.194 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:26.194 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:26.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:26.280 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:26.274+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:26.315 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:26.315 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:26.315 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:26.315 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:26.315 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:26.325 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:26.381 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:26.378+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189765 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1406, mlcod=27'1)", 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:26.432 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.77490278, 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001809768, 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653436+0000", 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3040000000000005e-06 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653452+0000", 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6333000000000001e-05 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653463+0000", 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0574000000000001e-05 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653574+0000", 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110859 2024-09-13T09:29:26.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653617+0000", 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3325999999999998e-05 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655197+0000", 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015795729999999999 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655238+0000", 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1390999999999997e-05 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189786 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1409, mlcod=27'1)", 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.746392932999999, 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049715810000000001, 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.437 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681937+0000", 2024-09-13T09:29:26.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681955+0000", 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7588000000000002e-05 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681979+0000", 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4073000000000001e-05 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681988+0000", 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2539999999999998e-06 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682028+0000", 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0203000000000001e-05 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682044+0000", 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5586e-05 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686830+0000", 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047863819999999996 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686910+0000", 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9740999999999994e-05 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.441 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189812 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1413, mlcod=27'1)", 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.711291605, 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001885103, 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717038+0000", 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717050+0000", 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2282e-05 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717070+0000", 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9337000000000001e-05 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717087+0000", 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983e-05 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717148+0000", 2024-09-13T09:29:26.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1765000000000002e-05 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717183+0000", 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4646e-05 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718857+0000", 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00167435 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718925+0000", 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189821 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1415, mlcod=27'1)", 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.697715617, 2024-09-13T09:29:26.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018352900000000001, 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730614+0000", 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730627+0000", 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2531000000000001e-05 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730652+0000", 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5663000000000001e-05 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730673+0000", 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0222000000000001e-05 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730779+0000", 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010659500000000001 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730822+0000", 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3182000000000001e-05 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732358+0000", 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015359309999999999 2024-09-13T09:29:26.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732451+0000", 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2554999999999997e-05 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189823 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1416, mlcod=27'1)", 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.694202969999999, 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017908609999999999, 2024-09-13T09:29:26.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734127+0000", 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734140+0000", 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2886e-05 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734163+0000", 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3266999999999999e-05 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734179+0000", 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6552e-05 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734270+0000", 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-13T09:29:26.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734315+0000", 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4555999999999998e-05 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735878+0000", 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015635079999999999 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735919+0000", 2024-09-13T09:29:26.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0917999999999997e-05 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189827 3.1c e193/191 3:3831950c:::rbd_directory:head v 193'784, mlcod=30'142)", 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.684903169, 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001751297, 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743426+0000", 2024-09-13T09:29:26.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743445+0000", 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8576e-05 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743504+0000", 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8888999999999998e-05 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743521+0000", 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743640+0000", 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119781 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743689+0000", 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8134999999999999e-05 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745092+0000", 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014033419999999999 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745179+0000", 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7398000000000006e-05 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189844 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1417, mlcod=27'1)", 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.660694662000001, 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018671510000000001, 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767635+0000", 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767648+0000", 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3448000000000001e-05 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767681+0000", 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2508000000000001e-05 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767705+0000", 2024-09-13T09:29:26.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4511e-05 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767771+0000", 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5772999999999995e-05 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767825+0000", 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3628999999999998e-05 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769445+0000", 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016198460000000001 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769504+0000", 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9043999999999999e-05 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189857 3.14 e193/191 3:283508b2:::rbd_data.1118fb4e889d.0000000000000002:head v 193'607, mlcod=0'0)", 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.644329323999999, 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030577590000000002, 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:26.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784000+0000", 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784015+0000", 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4663e-05 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784043+0000", 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8521999999999999e-05 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784056+0000", 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2591e-05 2024-09-13T09:29:26.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784142+0000", 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6336999999999997e-05 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784196+0000", 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4323999999999997e-05 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.786969+0000", 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027724210000000002 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787060+0000", 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0841999999999995e-05 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189862 3.13 e193/191 3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head v 193'1112, mlcod=26'1)", 2024-09-13T09:29:26.470 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.642736275000001, 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00238909, 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785594+0000", 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785603+0000", 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785615+0000", 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2490999999999999e-05 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785623+0000", 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785736+0000", 2024-09-13T09:29:26.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011302999999999999 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785788+0000", 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1433e-05 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787873+0000", 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020855499999999998 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787984+0000", 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110661 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189944 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.528828729000001, 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.77023277800000001, 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:26.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189944 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899501+0000", 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899511+0000", 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9010000000000006e-06 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899528+0000", 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7445999999999999e-05 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899545+0000", 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899618+0000", 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2534999999999996e-05 2024-09-13T09:29:26.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899627+0000", 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.667960+0000", 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.76833267900000002 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.668076+0000", 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.669735+0000", 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587539999999999 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.3 193/191 [PushOp(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6, version: 30'16, data_included: [1126604~6238], data_size: 6238, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6@30'16, size: 10485760, copy_subset: [0~10485760], clone_subset: {}, snapset: 6=[]:{6=[6]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:10485760, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:26.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.317787032, 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017837389999999999, 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110543+0000", 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110579+0000", 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5716000000000002e-05 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110617+0000", 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8492999999999999e-05 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110646+0000", 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9224999999999999e-05 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110840+0000", 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019405299999999999 2024-09-13T09:29:26.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110862+0000", 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2195999999999999e-05 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.112328+0000", 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014654100000000001 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e193,191 3:2c83699f:::rbd_data.111817fd4ca8.000000000000011a:head,68'129)", 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.768136420999999, 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020817029999999999, 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:26.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660193+0000", 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660204+0000", 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660225+0000", 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0594999999999998e-05 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660242+0000", 2024-09-13T09:29:26.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-05 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662101+0000", 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018583860000000001 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662117+0000", 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5886999999999999e-05 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662276+0000", 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159901 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e194,191 3:2dffdd96:::rbd_data.111817fd4ca8.0000000000000162:head,68'133)", 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.763523230000001, 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021107690000000002, 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:26.489 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664807+0000", 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664814+0000", 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664829+0000", 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5130000000000001e-05 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664844+0000", 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4782e-05 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665589+0000", 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074510899999999998 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665598+0000", 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9199999999999993e-06 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666919+0000", 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00132099 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(osd.7.0:5052 3.f e194/193 3:f5427e30:::rbd_data.11189fb5ddd6.0000000000000003:b v 194'480, mlcod=0'0)", 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.761889991, 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060970809999999999, 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.493 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666447+0000", 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666455+0000", 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666461+0000", 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666496+0000", 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5063e-05 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666515+0000", 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:26.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672509+0000", 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059937419999999998 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672538+0000", 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9645000000000002e-05 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189946 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.755294361000001, 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068095500000000002, 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189946 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673035+0000", 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:26.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673044+0000", 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673060+0000", 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5872999999999999e-05 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673068+0000", 2024-09-13T09:29:26.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1759999999999993e-06 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673112+0000", 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3998000000000001e-05 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673189+0000", 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6330999999999998e-05 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673305+0000", 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011642 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.678066+0000", 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047612519999999997 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679714+0000", 2024-09-13T09:29:26.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001647665 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679755+0000", 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0941e-05 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679846+0000", 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1558000000000007e-05 2024-09-13T09:29:26.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190167 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.154606955, 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024116239999999998, 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:26.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190167 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:26.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273722+0000", 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273736+0000", 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.345e-05 2024-09-13T09:29:26.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273767+0000", 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1359000000000002e-05 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273783+0000", 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5549999999999999e-05 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273893+0000", 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274045+0000", 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015171299999999999 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274391+0000", 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034663899999999999 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.275713+0000", 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013217630000000001 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276007+0000", 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029441100000000001 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276074+0000", 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276136+0000", 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1420999999999994e-05 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190169 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:26.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.150220373, 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029159210000000001, 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190169 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278109+0000", 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278124+0000", 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5449000000000001e-05 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278157+0000", 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2602999999999999e-05 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278178+0000", 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1195999999999998e-05 2024-09-13T09:29:26.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278298+0000", 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278456+0000", 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015802899999999999 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:26.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278817+0000", 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036107899999999999 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280193+0000", 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001375392 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280781+0000", 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058817699999999995 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280865+0000", 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3697999999999997e-05 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281027+0000", 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016203300000000001 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.516 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190170 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.147202581, 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023661509999999999, 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190170 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281127+0000", 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281140+0000", 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281164+0000", 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4369999999999999e-05 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281182+0000", 2024-09-13T09:29:26.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8016000000000002e-05 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281324+0000", 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014187999999999999 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281446+0000", 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121198 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281690+0000", 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024418399999999999 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283024+0000", 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013344030000000001 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283315+0000", 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029132399999999998 2024-09-13T09:29:26.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283392+0000", 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6452000000000006e-05 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283495+0000", 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010284400000000001 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190173 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.143293164999999, 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025929310000000001, 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190173 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285036+0000", 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285047+0000", 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1083e-05 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285070+0000", 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3050000000000001e-05 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285082+0000", 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1749999999999999e-05 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285169+0000", 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6514000000000006e-05 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285332+0000", 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016333 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285618+0000", 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028579000000000001 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.286946+0000", 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001327679 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287517+0000", 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00057192199999999999 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287571+0000", 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3903000000000002e-05 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287631+0000", 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.961e-05 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:190198 3.11 e198/197 3:8daa9abf:::rbd_data.11189888a71f.0000000000000001:head v 198'880, mlcod=193'877)", 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.11097034, 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023753540000000001, 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:26.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317359+0000", 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:26.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317375+0000", 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6348999999999999e-05 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317416+0000", 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0562000000000002e-05 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317432+0000", 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5957e-05 2024-09-13T09:29:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317658+0000", 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000225768 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317716+0000", 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7819000000000003e-05 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319644+0000", 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019285999999999999 2024-09-13T09:29:26.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319736+0000", 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2034999999999996e-05 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:26.532 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:26.534 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:26.532+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:26.534 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:26.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:26.635 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:26.630+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:26.641 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:26.641 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:26.642 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:26.642 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:26.652 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:26.687 INFO:teuthology.orchestra.run.smithi063.stdout:3.9 2024-09-13T09:29:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:3.4 2024-09-13T09:29:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:3.1a 2024-09-13T09:29:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:3.18 2024-09-13T09:29:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:3.13 2024-09-13T09:29:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:3.0 2024-09-13T09:29:26.688 INFO:teuthology.orchestra.run.smithi063.stdout:1.5 2024-09-13T09:29:26.735 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:26.730+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:26.753 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:26.753 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:26.753 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:26.753 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:26.753 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:26.764 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:26.776 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:26.833 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:26.835 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:26.832+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:26.858 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:26.894 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:26.896 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:26.897 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:26.907 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:26.921 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:26.922 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:26.925 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:26.925 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:26.969 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:26.966+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:27.025 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.026 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:27.036 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:27.034+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:27.137 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:27.134+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:27.146 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.146 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:27.238 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:27.236+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:27.294 INFO:tasks.thrashosds.thrasher:Moving pg 3.17 from osd.5 to osd.2 2024-09-13T09:29:27.294 INFO:tasks.thrashosds.thrasher:Transfer export file from ubuntu@smithi099.front.sepia.ceph.com to ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:29:27.294 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-09-13T09:29:27.310 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-09-13T09:29:27.322 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-13T09:29:27.325 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.325 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:27.339 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:27.334+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:27.359 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.ZwEfECHr0M 2024-09-13T09:29:27.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /var/log/ceph/exp.3.17.5 /tmp/tmp.ZwEfECHr0M 2024-09-13T09:29:27.375 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.ZwEfECHr0M 2024-09-13T09:29:27.400 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.ZwEfECHr0M is 1MB 2024-09-13T09:29:27.443 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:27.438+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:27.469 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.469 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.471 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.ZwEfECHr0M 2024-09-13T09:29:27.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:27.549 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:27.546+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:27.550 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.17.5 2024-09-13T09:29:27.591 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:29:27.632 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.632 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:27.644 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:27.642+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:27.745 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:27.740+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:27.745 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.746 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.755 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:27.846 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:27.842+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:27.857 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.857 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:27.947 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:27.942+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:27.969 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:27.969 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:27.979 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:28.048 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:28.044+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:28.083 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.083 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.092 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:28.149 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:28.144+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:28.204 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.204 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.214 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:28.249 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:28.248+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:28.320 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.321 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.330 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:28.350 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:28.348+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:28.442 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.443 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.451 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:28.446+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:28.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:28.552 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:28.550+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:28.562 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:29:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.568 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:28.653 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:28.650+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:28.681 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.682 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.691 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:28.746 INFO:teuthology.orchestra.run.smithi063.stdout:get_pg_num_history pg_num_history pg_num_history(e194 pg_nums {1={16=8},2={17=1},3={21=32}} deleted_pools ) 2024-09-13T09:29:28.747 INFO:teuthology.orchestra.run.smithi063.stdout:Importing pgid 3.17 2024-09-13T09:29:28.754 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:e89855de:::rbd_object_map.1118e98b86fc:head# 2024-09-13T09:29:28.754 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:29:28.754 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:28.752+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:28.758 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:eaa625ad:::rbd_data.111867511a81.0000000000000010:head# 2024-09-13T09:29:28.758 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 5=[]:{} 2024-09-13T09:29:28.760 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:eb7ede79:::rbd_data.1118a28cd5c4.00000000000000ca:head# 2024-09-13T09:29:28.761 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:29:28.763 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:ec7c1922:::rbd_data.1118a28cd5c4.000000000000008b:head# 2024-09-13T09:29:28.764 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:29:28.766 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:ed4f9a1c:::rbd_data.111888f95246.000000000000000f:head# 2024-09-13T09:29:28.766 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 2e=[]:{} 2024-09-13T09:29:28.769 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:ed4f9ac5:::rbd_data.1118a28cd5c4.000000000000002d:head# 2024-09-13T09:29:28.769 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:29:28.772 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:ed7fd31c:::rbd_object_map.11185454a1c8:head# 2024-09-13T09:29:28.772 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 0=[]:{} 2024-09-13T09:29:28.774 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:ee110ec6:::rbd_data.111867511a81.0000000000000003:5# 2024-09-13T09:29:28.777 INFO:teuthology.orchestra.run.smithi063.stdout:Write #3:ee110ec6:::rbd_data.111867511a81.0000000000000003:head# 2024-09-13T09:29:28.777 INFO:teuthology.orchestra.run.smithi063.stdout:snapset 5=[]:{5=[5]} 2024-09-13T09:29:28.781 INFO:teuthology.orchestra.run.smithi063.stdout:write_pg epoch 179 info 3.17( v 177'761 (0'0,177'761] local-lis/les=60/61 n=9 ec=21/21 lis/c=60/60 les/c/f=61/61/0 sis=60) 2024-09-13T09:29:28.795 INFO:teuthology.orchestra.run.smithi063.stdout:Import successful 2024-09-13T09:29:28.796 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.807 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:28.855 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:28.850+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:28.922 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:28.922 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:28.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:28.933 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:28.956 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:28.950+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:29.035 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:29.035 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:29.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:29.068 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:29.069 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:29.066+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:29.089 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:29.109 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:29.131 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:29.154 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:29.157 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:29.154+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 167.61075932200001, 2024-09-13T09:29:29.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:29.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:29.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 135.023100943, 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:29.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.187 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 82.892808818000006, 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:29.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 82.308598367000002, 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:29.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:29.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.196 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907264734999998, 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.197 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:29.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907246727, 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:29.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:29.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:29.206 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907229388000005, 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:29.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907212130999994, 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.211 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.214 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.215 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907194594999993, 2024-09-13T09:29:29.216 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.218 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.219 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.220 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907177524999994, 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.221 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.222 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:29.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.224 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:29.225 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907163866000005, 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.226 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.228 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:29.229 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.230 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907154192999997, 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.231 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:29.233 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.234 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:29.235 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907139647999998, 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:29.236 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.237 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.238 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:29.239 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907122981000001, 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.240 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:29.241 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.242 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:29.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.244 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.907105889999997, 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.245 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:29.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.249 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.905250314, 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.905234206000003, 2024-09-13T09:29:29.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:29.258 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.905219841999994, 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.259 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.263 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.905206014000001, 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.267 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:29.267 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.267 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:29.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:29.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.268 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 81.905188314, 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.269 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:29.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.272 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.273 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.274 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.274 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.274 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.274 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:29.274 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:29.275 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:29.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:29.317 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:29.314+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:29.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /var/log/ceph/exp.3.17.5 2024-09-13T09:29:29.363 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /var/log/ceph/exp.3.17.5 2024-09-13T09:29:29.379 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:29.374+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:29.435 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:29.454 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:29.454 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:29.455 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:29.501 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:29.498+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:29.567 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:29.567 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:29.567 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:29.567 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:29.567 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:29.577 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:29.580 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:29.576+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:29.681 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:29.680+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:29.687 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107276832, 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.688 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:29.689 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.690 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:29.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107264595, 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.693 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:29.695 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.695 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.696 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.696 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:29.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.696 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:29.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.10724919800001, 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.699 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:29.700 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.701 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.702 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.703 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107234323, 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.704 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:29.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.10721711399999, 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.708 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:29.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.712 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107196659, 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.713 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.10717857900001, 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:29.717 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:29.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:29.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.721 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107162203, 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.722 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:29.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107146852, 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:29.726 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:29.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.730 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107130148, 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.731 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:29.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:29.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.736 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.736 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:29.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:29.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:29.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107112066, 2024-09-13T09:29:29.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:29.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.107095321, 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.743 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:29.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.745 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.747 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.10705066200001, 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.748 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:29.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.752 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.104101222, 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:29.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.754 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:29.756 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.104080684, 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.757 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:29.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.761 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.104058069, 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:29.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:29.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.10403920100001, 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.766 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:29.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.770 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.10400460699999, 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:29.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.774 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.099725702, 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:29.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:29.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 102.09966172999999, 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:29.779 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:29.783 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:29.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:29.785 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:29.782+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:29.886 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:29.882+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:29.905 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:29.906 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:29.906 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:29.906 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:29.916 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:29.987 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:29.982+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:30.021 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:30.021 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:30.021 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:30.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:30.088 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:30.082+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:30.136 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:30.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.708189042000001, 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:30.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.708173986000006, 2024-09-13T09:29:30.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:30.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.146 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.708164577999995, 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:30.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:30.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:30.154 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.708150285000002, 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:30.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.708064578999995, 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.160 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:30.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707708330000003, 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.165 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:30.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:30.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707661494000007, 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.171 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:30.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:30.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.176 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707622508, 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:30.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:30.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:30.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:30.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707602363999996, 2024-09-13T09:29:30.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:30.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707583295999996, 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.188 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:30.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.192 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707541207000006, 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:30.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.198 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707427705000001, 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:30.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707409526000006, 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:30.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.208 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707399781000007, 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:30.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707389594000006, 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707296728000003, 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:30.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:30.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.225 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.707193316000001, 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:30.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.706871437999993, 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:30.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:30.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.236 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.706455496999993, 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:30.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:30.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.706201786999998, 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:30.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.247 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:30.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:30.249 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:30.244+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:30.350 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:30.348+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:30.359 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:30.359 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:30.359 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:30.359 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:30.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:30.451 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:30.446+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:30.470 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:30.470 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:30.470 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:30.470 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:30.470 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:30.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:30.553 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:30.550+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:30.577 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:29:30.588 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:30.588 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:30.589 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 227.62316154600001, 2024-09-13T09:29:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:30.590 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:30.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:30.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:30.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 164.50994180199999, 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:30.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 164.50954378700001, 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:30.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.83443691100001, 2024-09-13T09:29:30.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:30.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:30.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.83442197900001, 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.83438887299999, 2024-09-13T09:29:30.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:30.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.83437258000001, 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:30.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.83435646500001, 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:30.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:30.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.635 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.83432542400001, 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.636 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:30.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:30.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.834293801, 2024-09-13T09:29:30.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:30.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.82134574, 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:30.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.82132868400001, 2024-09-13T09:29:30.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:30.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:30.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.82131251600001, 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:30.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:30.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.663 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.82109885400001, 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.821079543, 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.82106314999999, 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:30.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.821047488, 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:30.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:30.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:30.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 163.82102113400001, 2024-09-13T09:29:30.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:30.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 117.292508407, 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:30.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:30.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:30.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:30.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 112.18059429199999, 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:30.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:30.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.701 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:30.701 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:30.701 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:30.702 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:30.703 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:30.698+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:30.803 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:30.798+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:30.811 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:30.811 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:30.811 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:30.821 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:30.904 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:30.902+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:30.925 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:30.925 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:30.926 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:30.926 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:30.926 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:30.936 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:31.006 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:31.002+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:31.088 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:31.089 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:31.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:31.089 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:31.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.089 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:31.090 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:31.090 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 146.34655484699999, 2024-09-13T09:29:31.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:31.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.090 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:31.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.091 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.091 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:31.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.091 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:31.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:31.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:31.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:31.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:31.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:31.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:31.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:31.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:31.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:31.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:31.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:31.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:31.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:31.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:31.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:31.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:31.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:31.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:31.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:31.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:31.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:31.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:31.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:31.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:31.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:31.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:31.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:31.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:31.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:31.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:31.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:31.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:31.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.105 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:31.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.104382095999995, 2024-09-13T09:29:31.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:31.107 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.107 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:31.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.108 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:31.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:31.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:31.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:31.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:31.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:31.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:31.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:31.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:31.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:31.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:31.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:31.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:31.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:31.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:31.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.116 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.116 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.117 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.117 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:31.117 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.104358880999996, 2024-09-13T09:29:31.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:31.117 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.118 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.118 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.118 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:31.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.119 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:31.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:31.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:31.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:31.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:31.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:31.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:31.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.124 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:31.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:31.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:31.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:31.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:31.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:31.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.127 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.128 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:31.128 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.100230564, 2024-09-13T09:29:31.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:31.128 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.128 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:31.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.130 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:31.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:31.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.100178004, 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.135 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:31.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:31.139 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.100141308999994, 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.140 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:31.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:31.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:31.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:31.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:31.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.147 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.100121678999997, 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.148 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:31.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.100099499999999, 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.156 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.096343716999996, 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.157 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:31.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:31.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.096318382999996, 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.162 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:31.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:31.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.096300545999995, 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:31.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.173 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.096288161000004, 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.174 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.175 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:31.176 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.177 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.096259399999994, 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.178 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:31.179 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.180 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.181 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.182 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.092107189000004, 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.183 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.184 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:31.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:31.186 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.092078900999994, 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.187 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.188 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:31.189 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.190 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:31.191 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.092052766999998, 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:31.192 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.193 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.194 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:31.195 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.092028130000003, 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.196 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.197 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:31.198 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.199 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.200 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.091945555999999, 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:31.201 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.202 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.203 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.204 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.091902653000005, 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:31.205 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.206 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.207 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:31.208 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 85.088276930999996, 2024-09-13T09:29:31.209 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.210 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:31.211 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:31.212 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.213 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:31.214 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:31.214 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:31.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:31.216 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:31.210+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:31.276 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:31.316 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:31.314+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:31.332 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:31.342 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:31.417 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:31.412+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:31.433 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:31.444 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:31.445 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:31.445 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:31.445 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:31.445 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:31.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:31.455 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:31.477 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:31.500 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:31.517 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:31.516+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:31.523 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:31.547 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:31.547 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:31.548 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:31.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:31.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:31.561 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:31.561 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:31.571 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:31.618 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:31.616+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:31.656 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:29:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:31.691 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:31.701 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:31.719 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:31.716+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:31.804 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:31.804 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:31.814 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:31.820 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:31.818+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:31.921 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:31.918+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:31.927 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:32.022 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:32.020+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:32.035 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.036 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:32.123 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:32.120+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:32.155 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.155 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:32.224 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:32.220+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:32.276 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.276 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:32.334 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:32.332+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:32.406 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.407 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:32.425 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:32.422+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:32.524 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.524 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.525 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:32.522+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:32.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:32.627 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:32.624+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:32.641 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.641 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:32.728 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:32.724+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:32.804 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.804 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:32.829 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:32.828+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:32.921 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:32.922 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:32.930 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:32.928+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:32.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:33.031 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:33.028+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:33.083 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:33.083 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:33.093 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:33.132 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:33.128+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:33.212 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:33.213 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:33.223 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:33.233 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:33.232+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:33.332 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:33.332 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:33.334 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:33.330+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:33.342 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:33.436 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:33.430+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:33.449 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:33.449 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:33.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:33.537 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:33.532+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:33.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:33.573 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:33.573 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:33.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:33.638 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:33.636+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:33.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:29:33.693 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:33.693 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:33.703 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:33.706 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:33.728 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:33.746 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:33.744+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:33.749 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:33.770 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:33.793 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:33.821 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:33.822 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:33.823 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:33.824 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:33.840 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:33.836+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:33.859 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:33.859 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189765 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1406, mlcod=27'1)", 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.202390959999999, 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001809768, 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:33.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653436+0000", 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3040000000000005e-06 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653452+0000", 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6333000000000001e-05 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653463+0000", 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0574000000000001e-05 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653574+0000", 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110859 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653617+0000", 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3325999999999998e-05 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655197+0000", 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015795729999999999 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655238+0000", 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1390999999999997e-05 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189786 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1409, mlcod=27'1)", 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.173881113, 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049715810000000001, 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:33.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681937+0000", 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681955+0000", 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7588000000000002e-05 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681979+0000", 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4073000000000001e-05 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681988+0000", 2024-09-13T09:29:33.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2539999999999998e-06 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682028+0000", 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0203000000000001e-05 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682044+0000", 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5586e-05 2024-09-13T09:29:33.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686830+0000", 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047863819999999996 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686910+0000", 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9740999999999994e-05 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189812 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1413, mlcod=27'1)", 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.138779785000001, 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001885103, 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717038+0000", 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717050+0000", 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2282e-05 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717070+0000", 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9337000000000001e-05 2024-09-13T09:29:33.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717087+0000", 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983e-05 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717148+0000", 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1765000000000002e-05 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717183+0000", 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4646e-05 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718857+0000", 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00167435 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718925+0000", 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189821 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1415, mlcod=27'1)", 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.125203797000001, 2024-09-13T09:29:33.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018352900000000001, 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730614+0000", 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730627+0000", 2024-09-13T09:29:33.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2531000000000001e-05 2024-09-13T09:29:33.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730652+0000", 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5663000000000001e-05 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730673+0000", 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0222000000000001e-05 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730779+0000", 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010659500000000001 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730822+0000", 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3182000000000001e-05 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732358+0000", 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015359309999999999 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732451+0000", 2024-09-13T09:29:33.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2554999999999997e-05 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189823 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1416, mlcod=27'1)", 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.12169115, 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017908609999999999, 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734127+0000", 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734140+0000", 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2886e-05 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734163+0000", 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3266999999999999e-05 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734179+0000", 2024-09-13T09:29:33.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6552e-05 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734270+0000", 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734315+0000", 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4555999999999998e-05 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735878+0000", 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015635079999999999 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735919+0000", 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0917999999999997e-05 2024-09-13T09:29:33.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189827 3.1c e193/191 3:3831950c:::rbd_directory:head v 193'784, mlcod=30'142)", 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.112391348999999, 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001751297, 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743426+0000", 2024-09-13T09:29:33.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743445+0000", 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8576e-05 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743504+0000", 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8888999999999998e-05 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743521+0000", 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743640+0000", 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119781 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743689+0000", 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8134999999999999e-05 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745092+0000", 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014033419999999999 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745179+0000", 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7398000000000006e-05 2024-09-13T09:29:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189844 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1417, mlcod=27'1)", 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.088182841999998, 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018671510000000001, 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767635+0000", 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767648+0000", 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3448000000000001e-05 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767681+0000", 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2508000000000001e-05 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767705+0000", 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4511e-05 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767771+0000", 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5772999999999995e-05 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767825+0000", 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3628999999999998e-05 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769445+0000", 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016198460000000001 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769504+0000", 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9043999999999999e-05 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189857 3.14 e193/191 3:283508b2:::rbd_data.1118fb4e889d.0000000000000002:head v 193'607, mlcod=0'0)", 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.071817503999998, 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030577590000000002, 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784000+0000", 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784015+0000", 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4663e-05 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784043+0000", 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8521999999999999e-05 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784056+0000", 2024-09-13T09:29:33.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2591e-05 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784142+0000", 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6336999999999997e-05 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784196+0000", 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4323999999999997e-05 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.786969+0000", 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027724210000000002 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787060+0000", 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0841999999999995e-05 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189862 3.13 e193/191 3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head v 193'1112, mlcod=26'1)", 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.070224455000002, 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00238909, 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:33.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785594+0000", 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785603+0000", 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785615+0000", 2024-09-13T09:29:33.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2490999999999999e-05 2024-09-13T09:29:33.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785623+0000", 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785736+0000", 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011302999999999999 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785788+0000", 2024-09-13T09:29:33.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1433e-05 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787873+0000", 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020855499999999998 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787984+0000", 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110661 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189944 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.956316909000002, 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.77023277800000001, 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:33.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189944 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899501+0000", 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899511+0000", 2024-09-13T09:29:33.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9010000000000006e-06 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899528+0000", 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7445999999999999e-05 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899545+0000", 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:29:33.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899618+0000", 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2534999999999996e-05 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899627+0000", 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.667960+0000", 2024-09-13T09:29:33.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.76833267900000002 2024-09-13T09:29:33.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.668076+0000", 2024-09-13T09:29:33.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:29:33.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.669735+0000", 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587539999999999 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.3 193/191 [PushOp(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6, version: 30'16, data_included: [1126604~6238], data_size: 6238, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6@30'16, size: 10485760, copy_subset: [0~10485760], clone_subset: {}, snapset: 6=[]:{6=[6]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:10485760, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:33.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.745275211999999, 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017837389999999999, 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:33.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110543+0000", 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110579+0000", 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5716000000000002e-05 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110617+0000", 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8492999999999999e-05 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110646+0000", 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9224999999999999e-05 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110840+0000", 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019405299999999999 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110862+0000", 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2195999999999999e-05 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.112328+0000", 2024-09-13T09:29:33.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014654100000000001 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e193,191 3:2c83699f:::rbd_data.111817fd4ca8.000000000000011a:head,68'129)", 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.195624600999999, 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020817029999999999, 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:33.919 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660193+0000", 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660204+0000", 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660225+0000", 2024-09-13T09:29:33.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0594999999999998e-05 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660242+0000", 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-05 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662101+0000", 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018583860000000001 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662117+0000", 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5886999999999999e-05 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662276+0000", 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159901 2024-09-13T09:29:33.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e194,191 3:2dffdd96:::rbd_data.111817fd4ca8.0000000000000162:head,68'133)", 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.191011410000002, 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021107690000000002, 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:33.925 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664807+0000", 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664814+0000", 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:29:33.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664829+0000", 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5130000000000001e-05 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664844+0000", 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4782e-05 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665589+0000", 2024-09-13T09:29:33.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074510899999999998 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665598+0000", 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9199999999999993e-06 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666919+0000", 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00132099 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(osd.7.0:5052 3.f e194/193 3:f5427e30:::rbd_data.11189fb5ddd6.0000000000000003:b v 194'480, mlcod=0'0)", 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.189378171000001, 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060970809999999999, 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:33.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666447+0000", 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666455+0000", 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666461+0000", 2024-09-13T09:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666496+0000", 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5063e-05 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666515+0000", 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672509+0000", 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059937419999999998 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672538+0000", 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9645000000000002e-05 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189946 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 21.182782541000002, 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068095500000000002, 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189946 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673035+0000", 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:33.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673044+0000", 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673060+0000", 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5872999999999999e-05 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673068+0000", 2024-09-13T09:29:33.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1759999999999993e-06 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673112+0000", 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3998000000000001e-05 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673189+0000", 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6330999999999998e-05 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673305+0000", 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011642 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.678066+0000", 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047612519999999997 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679714+0000", 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001647665 2024-09-13T09:29:33.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679755+0000", 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0941e-05 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679846+0000", 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1558000000000007e-05 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190167 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.582095134999999, 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024116239999999998, 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190167 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273722+0000", 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273736+0000", 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.345e-05 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273767+0000", 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1359000000000002e-05 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273783+0000", 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5549999999999999e-05 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273893+0000", 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274045+0000", 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015171299999999999 2024-09-13T09:29:33.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274391+0000", 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034663899999999999 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.275713+0000", 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013217630000000001 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276007+0000", 2024-09-13T09:29:33.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029441100000000001 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276074+0000", 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276136+0000", 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1420999999999994e-05 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190169 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.577708553000001, 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029159210000000001, 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190169 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278109+0000", 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278124+0000", 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5449000000000001e-05 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278157+0000", 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2602999999999999e-05 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278178+0000", 2024-09-13T09:29:33.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1195999999999998e-05 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278298+0000", 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278456+0000", 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015802899999999999 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:33.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278817+0000", 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036107899999999999 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280193+0000", 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001375392 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280781+0000", 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058817699999999995 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280865+0000", 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3697999999999997e-05 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281027+0000", 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016203300000000001 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190170 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:33.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.574690760999999, 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023661509999999999, 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190170 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:33.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281127+0000", 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281140+0000", 2024-09-13T09:29:33.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281164+0000", 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4369999999999999e-05 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281182+0000", 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8016000000000002e-05 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281324+0000", 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014187999999999999 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281446+0000", 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121198 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281690+0000", 2024-09-13T09:29:33.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024418399999999999 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283024+0000", 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013344030000000001 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283315+0000", 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029132399999999998 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283392+0000", 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6452000000000006e-05 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283495+0000", 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010284400000000001 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190173 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:33.959 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.570781345, 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025929310000000001, 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190173 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:33.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285036+0000", 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285047+0000", 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1083e-05 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285070+0000", 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3050000000000001e-05 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285082+0000", 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1749999999999999e-05 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285169+0000", 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6514000000000006e-05 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285332+0000", 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016333 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285618+0000", 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028579000000000001 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.286946+0000", 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001327679 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287517+0000", 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00057192199999999999 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287571+0000", 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3903000000000002e-05 2024-09-13T09:29:33.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287631+0000", 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.961e-05 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:190198 3.11 e198/197 3:8daa9abf:::rbd_data.11189888a71f.0000000000000001:head v 198'880, mlcod=193'877)", 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.538458519999999, 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023753540000000001, 2024-09-13T09:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317359+0000", 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317375+0000", 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6348999999999999e-05 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317416+0000", 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0562000000000002e-05 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317432+0000", 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5957e-05 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317658+0000", 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000225768 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317716+0000", 2024-09-13T09:29:33.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7819000000000003e-05 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319644+0000", 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019285999999999999 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319736+0000", 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2034999999999996e-05 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:33.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:33.970 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:33.971 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:33.968+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:33.971 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:34.072 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:34.070+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:34.082 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:34.082 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:34.082 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:34.082 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:34.092 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:34.173 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:34.168+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:34.198 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:34.198 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:34.198 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:34.198 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:34.198 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:34.208 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:34.274 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:34.272+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 172.750582248, 2024-09-13T09:29:34.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:34.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.322 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 140.162923869, 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:34.324 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:34.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.329 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 88.032631744, 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.331 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:34.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.448421292999996, 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.334 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.336 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.337 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:34.338 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.047087661000006, 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.339 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:34.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.047069652999994, 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.344 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:34.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:34.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:34.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.348 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.047052313999998, 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.349 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.352 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.047035057000002, 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.353 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.354 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:34.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:34.357 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.047017521000001, 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:34.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.361 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.047000451000002, 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:34.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:34.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.046986791999998, 2024-09-13T09:29:34.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:34.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.370 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.046977119000005, 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.371 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:34.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.046962574000005, 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.376 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.377 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:34.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:34.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:34.380 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.046945906999994, 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:34.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.382 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.384 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.046928816000005, 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:34.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.387 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.045073239999994, 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.389 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.045057131999997, 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:34.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:34.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:34.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.045042768000002, 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.398 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:34.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.402 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.045028939999995, 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:34.406 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 87.045011239999994, 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:34.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.411 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:34.412 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 6, 7, 0, 2, 4] out_osds: [] dead_osds: [5, 2] live_osds: [0, 1, 6, 7, 3, 4] 2024-09-13T09:29:34.412 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:29:34.412 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-13T09:29:34.412 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T09:29:34.412 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T09:29:34.414 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:34.414 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:34.410+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:34.416 INFO:tasks.ceph.osd.2:Started 2024-09-13T09:29:34.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:34.514 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:34.512+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:34.519 INFO:teuthology.orchestra.run.smithi063.stderr:no valid command found; 10 closest matches: 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:0 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:1 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:2 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:abort 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:assert 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:config diff 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:config diff get 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:config get 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:config help [] 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:config set ... 2024-09-13T09:29:34.520 INFO:teuthology.orchestra.run.smithi063.stderr:admin_socket: invalid command 2024-09-13T09:29:34.521 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:34.521 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:34.521 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:34.521 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:34.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:29:34.522 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-13T09:29:34.533 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:34.615 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:34.610+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:34.638 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:34.639 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:34.639 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:34.639 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:34.639 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:34.650 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:34.669 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:29:34.716 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:34.712+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179437859, 2024-09-13T09:29:34.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.762 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.763 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.764 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179425622, 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.765 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:34.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.769 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179410225, 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:34.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.17939534999999, 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.774 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:34.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.776 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.778 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179378141, 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:34.782 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179357686, 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.783 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:34.786 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179339606, 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.787 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.788 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.789 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:34.790 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.791 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.17932322999999, 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.792 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.793 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:34.794 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:34.795 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.17930787900001, 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.796 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.797 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:34.798 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.799 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:34.800 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179291175, 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.801 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:34.802 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.803 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.804 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.17927309300001, 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.805 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.806 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:34.807 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:34.808 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:34.809 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179256348, 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.810 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:34.811 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.812 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.813 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.179211689, 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.814 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.815 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.816 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.817 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:34.818 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.176262249, 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.819 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.820 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.821 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.822 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.176241711, 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:34.823 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.825 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.826 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.176219096, 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.827 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:34.830 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.831 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.176200228, 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.834 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.835 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.836 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.176165634, 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.837 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.838 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.839 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:34.840 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.17188672899999, 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:34.841 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.842 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.843 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.844 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.845 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 107.171822757, 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:34.846 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.847 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.848 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:34.849 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:34.850 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:34.850 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:34.851 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:34.762+0000 7f78c6df1880 -1 Falling back to public interface 2024-09-13T09:29:34.851 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:34.852 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:34.848+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:34.954 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:34.950+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:34.967 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:34.967 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:34.967 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:34.967 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:34.977 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:35.054 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:35.052+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:35.081 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:35.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:35.081 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:35.082 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:35.082 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:35.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:35.155 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:35.152+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768921700000007, 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:35.197 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.202 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768906643999998, 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:35.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768897236000001, 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:35.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768882942999994, 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.212 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:35.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768797237000001, 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.217 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:35.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:35.221 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768440987999995, 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:35.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:35.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768394151999999, 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.230 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768355166000006, 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768335022000002, 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768315954000002, 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768273864999998, 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768160363000007, 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768142183999998, 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768132438999999, 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768122251999998, 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.768029385999995, 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.767925973999994, 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.767604095999999, 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.767188154999999, 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.766934445000004, 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.294 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:35.295 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:35.295 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:35.290+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:35.396 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:35.390+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:35.408 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:35.408 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:35.408 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:35.408 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:35.418 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:35.496 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:35.494+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:35.522 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:35.522 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:35.522 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:35.522 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:35.522 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:35.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:35.598 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:35.594+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:35.639 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:35.639 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:35.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:35.639 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 232.67428006399999, 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:35.640 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:35.641 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:35.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.641 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:35.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:35.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.645 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 169.56106032, 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:35.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:35.650 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 169.56066230499999, 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:35.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:35.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.88555542899999, 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:35.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:35.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.88554049699999, 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:35.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:35.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:35.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:35.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:35.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:35.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:35.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:35.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.885507391, 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:35.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:35.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.88549109799999, 2024-09-13T09:29:35.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:35.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.88547498299999, 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.678 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.88544394199999, 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:35.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.88541231900001, 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:35.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:35.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87246425800001, 2024-09-13T09:29:35.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:35.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87244720199999, 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87243103399999, 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87221737199999, 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:35.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:35.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87219806100001, 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:35.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:35.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.872181668, 2024-09-13T09:29:35.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:35.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.717 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87216600599999, 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:35.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:35.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 168.87213965199999, 2024-09-13T09:29:35.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:35.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:35.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.343626925, 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:35.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:35.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:35.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:35.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 117.23171281, 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:35.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:35.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:35.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:35.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:35.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:35.739 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:35.741 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:35.741 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:35.738+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:35.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:35.842 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:35.840+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:35.859 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:35.859 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:35.859 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:35.859 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:35.870 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:35.943 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:35.938+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:35.973 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:35.976 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:35.976 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:35.977 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:35.977 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:35.977 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:35.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:35.995 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:36.018 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:36.042 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:36.043 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:36.040+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:36.067 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:36.090 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:36.091 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:36.095 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:36.095 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.096 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:36.154 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:36.152+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:36.227 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.227 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:36.245 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:36.244+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:36.339 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.339 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.346 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:36.344+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:36.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:36.449 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:36.446+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:36.497 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.497 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.507 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:36.553 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:36.550+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:36.625 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.625 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.636 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:36.650 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:36.646+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:36.683 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:29:36.744 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.744 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.751 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:36.748+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:36.754 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:36.852 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:36.848+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:36.863 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.863 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.872 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:36.953 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:36.948+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:36.976 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:36.976 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:36.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:37.054 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:37.050+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:37.090 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.091 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:37.155 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:37.152+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:37.256 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:37.250+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:37.258 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.258 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:37.357 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:37.354+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:37.385 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.385 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:37.458 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:37.454+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:37.501 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.502 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:37.559 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:37.554+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:37.613 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.613 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:37.660 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:37.658+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:37.694 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:29:37.730 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.730 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.741 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:37.760 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:37.758+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:37.844 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.845 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:37.861 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:37.858+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:37.962 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:37.962 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:37.962 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:37.958+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:37.971 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:38.063 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:38.058+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:38.096 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:38.098 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:38.098 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:38.109 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:38.164 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:38.162+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:38.187 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:38.182+0000 7f78c6df1880 -1 osd.2 194 log_to_monitors true 2024-09-13T09:29:38.214 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:38.214 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:38.224 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:38.231 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:38.253 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:38.265 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:38.260+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:38.271 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:38.301 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:38.328 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 153.595070459, 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:38.337 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.338 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:38.339 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.340 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:38.341 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.342 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:38.343 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.352897708, 2024-09-13T09:29:38.344 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.348 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.352874493000002, 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.349 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.350 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.351 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:38.352 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.353 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.348746176000006, 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.354 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.355 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:38.356 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.357 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.358 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.348693616000006, 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.359 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.360 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.361 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.362 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.348656921, 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.363 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.364 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:38.365 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.366 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.367 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.348637291000003, 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:38.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.369 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:38.370 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.371 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.372 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.348615112000005, 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:38.373 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.374 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.375 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:38.376 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.344859329000002, 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.377 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.378 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.379 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:38.380 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.381 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.344833995000002, 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:38.382 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:38.383 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.384 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:38.385 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.386 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.344816158, 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.387 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.388 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:38.389 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.390 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.391 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.344803772999995, 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.392 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.393 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.394 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:38.395 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:38.396 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.344775011999999, 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.397 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.398 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:38.399 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:38.400 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.401 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.340622800999995, 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.402 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.403 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:38.404 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.405 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.406 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.340594512999999, 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:38.407 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.408 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:38.409 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.410 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:38.411 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.340568379000004, 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:38.412 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.413 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:38.414 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:38.415 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.340543741999994, 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.416 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.417 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.418 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:38.419 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.420 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.340461168000004, 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:38.421 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.422 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.423 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:38.425 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.340418264999997, 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:38.426 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.427 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:38.428 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:38.429 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.336792543000001, 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:38.430 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:38.431 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:38.432 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.433 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:38.434 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:38.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:38.437 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:38.437 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:38.437 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:38.438 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:38.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:38.439 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:38.432+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:38.537 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:38.534+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:38.555 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:38.555 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:38.565 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:38.638 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:38.634+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:38.676 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:38.677 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:38.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:38.741 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:38.740+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:38.753 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:38.750+0000 7f78ba6df700 -1 osd.2 206 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189765 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1406, mlcod=27'1)", 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.132806804000001, 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001809768, 2024-09-13T09:29:38.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653428+0000", 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653436+0000", 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3040000000000005e-06 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653452+0000", 2024-09-13T09:29:38.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6333000000000001e-05 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653463+0000", 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0574000000000001e-05 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653574+0000", 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110859 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.653617+0000", 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3325999999999998e-05 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655197+0000", 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015795729999999999 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.655238+0000", 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1390999999999997e-05 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189786 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1409, mlcod=27'1)", 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.104296956999999, 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049715810000000001, 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681937+0000", 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681955+0000", 2024-09-13T09:29:38.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7588000000000002e-05 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681979+0000", 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4073000000000001e-05 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681988+0000", 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2539999999999998e-06 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682028+0000", 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0203000000000001e-05 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682044+0000", 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5586e-05 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686830+0000", 2024-09-13T09:29:38.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047863819999999996 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686910+0000", 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9740999999999994e-05 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189812 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1413, mlcod=27'1)", 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.069195628999999, 2024-09-13T09:29:38.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001885103, 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717040+0000", 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717038+0000", 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717050+0000", 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2282e-05 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717070+0000", 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9337000000000001e-05 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717087+0000", 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983e-05 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717148+0000", 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1765000000000002e-05 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.717183+0000", 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4646e-05 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718857+0000", 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00167435 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.718925+0000", 2024-09-13T09:29:38.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7206000000000007e-05 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189821 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1415, mlcod=27'1)", 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.055619641, 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018352900000000001, 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730616+0000", 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730614+0000", 2024-09-13T09:29:38.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730627+0000", 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2531000000000001e-05 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730652+0000", 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5663000000000001e-05 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730673+0000", 2024-09-13T09:29:38.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0222000000000001e-05 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730779+0000", 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010659500000000001 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.730822+0000", 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3182000000000001e-05 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732358+0000", 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015359309999999999 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.732451+0000", 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2554999999999997e-05 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189823 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1416, mlcod=27'1)", 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:38.808 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.052106993999999, 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017908609999999999, 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734128+0000", 2024-09-13T09:29:38.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734127+0000", 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734140+0000", 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2886e-05 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734163+0000", 2024-09-13T09:29:38.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3266999999999999e-05 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734179+0000", 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6552e-05 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734270+0000", 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0730999999999996e-05 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.734315+0000", 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4555999999999998e-05 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735878+0000", 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015635079999999999 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.735919+0000", 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0917999999999997e-05 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189827 3.1c e193/191 3:3831950c:::rbd_directory:head v 193'784, mlcod=30'142)", 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.042807193000002, 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001751297, 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743428+0000", 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743426+0000", 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743445+0000", 2024-09-13T09:29:38.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8576e-05 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743504+0000", 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8888999999999998e-05 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743521+0000", 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743640+0000", 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119781 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.743689+0000", 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8134999999999999e-05 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745092+0000", 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014033419999999999 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.745179+0000", 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7398000000000006e-05 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189844 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1417, mlcod=27'1)", 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.018598686000001, 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018671510000000001, 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767637+0000", 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767635+0000", 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767648+0000", 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3448000000000001e-05 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767681+0000", 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2508000000000001e-05 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767705+0000", 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4511e-05 2024-09-13T09:29:38.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767771+0000", 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5772999999999995e-05 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.767825+0000", 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3628999999999998e-05 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769445+0000", 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016198460000000001 2024-09-13T09:29:38.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.769504+0000", 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9043999999999999e-05 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189857 3.14 e193/191 3:283508b2:::rbd_data.1118fb4e889d.0000000000000002:head v 193'607, mlcod=0'0)", 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.002233348000001, 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030577590000000002, 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784000+0000", 2024-09-13T09:29:38.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784015+0000", 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4663e-05 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784043+0000", 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8521999999999999e-05 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784056+0000", 2024-09-13T09:29:38.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2591e-05 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784142+0000", 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6336999999999997e-05 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784196+0000", 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4323999999999997e-05 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.786969+0000", 2024-09-13T09:29:38.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027724210000000002 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787060+0000", 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0841999999999995e-05 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189862 3.13 e193/191 3:ca814742:::rbd_data.1118fb4e889d.0000000000000007:head v 193'1112, mlcod=26'1)", 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.000640299000001, 2024-09-13T09:29:38.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00238909, 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785595+0000", 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785594+0000", 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785603+0000", 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5220000000000007e-06 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785615+0000", 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2490999999999999e-05 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785623+0000", 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0940000000000008e-06 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785736+0000", 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011302999999999999 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.785788+0000", 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1433e-05 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787873+0000", 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020855499999999998 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787984+0000", 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110661 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189944 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-13T09:29:38.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.886732753, 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.77023277800000001, 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189944 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899501+0000", 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899511+0000", 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9010000000000006e-06 2024-09-13T09:29:38.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899528+0000", 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7445999999999999e-05 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899545+0000", 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899618+0000", 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2534999999999996e-05 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899627+0000", 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.667960+0000", 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.76833267900000002 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.668076+0000", 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.669735+0000", 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587539999999999 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGPush(3.3 193/191 [PushOp(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6, version: 30'16, data_included: [1126604~6238], data_size: 6238, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:c0f22b48:::rbd_data.1118474c503.0000000000000003:6@30'16, size: 10485760, copy_subset: [0~10485760], clone_subset: {}, snapset: 6=[]:{6=[6]}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:10485760, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:38.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.675691056000002, 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017837389999999999, 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110544+0000", 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110543+0000", 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110579+0000", 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5716000000000002e-05 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110617+0000", 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8492999999999999e-05 2024-09-13T09:29:38.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110646+0000", 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9224999999999999e-05 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110840+0000", 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019405299999999999 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.110862+0000", 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2195999999999999e-05 2024-09-13T09:29:38.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.112328+0000", 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014654100000000001 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e193,191 3:2c83699f:::rbd_data.111817fd4ca8.000000000000011a:head,68'129)", 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.126040445000001, 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020817029999999999, 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660195+0000", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660193+0000", 2024-09-13T09:29:38.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660204+0000", 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1096e-05 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660225+0000", 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0594999999999998e-05 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.660242+0000", 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7370000000000001e-05 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662101+0000", 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018583860000000001 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662117+0000", 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5886999999999999e-05 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.662276+0000", 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000159901 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "MOSDPGRecoveryDelete(3.14 e194,191 3:2dffdd96:::rbd_data.111817fd4ca8.0000000000000162:head,68'133)", 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.121427254, 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0021107690000000002, 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664808+0000", 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664807+0000", 2024-09-13T09:29:38.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664814+0000", 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3600000000000001e-06 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664829+0000", 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5130000000000001e-05 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.664844+0000", 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4782e-05 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665589+0000", 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074510899999999998 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.665598+0000", 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9199999999999993e-06 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666919+0000", 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00132099 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(osd.7.0:5052 3.f e194/193 3:f5427e30:::rbd_data.11189fb5ddd6.0000000000000003:b v 194'480, mlcod=0'0)", 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.119794015, 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060970809999999999, 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666447+0000", 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666455+0000", 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666461+0000", 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666496+0000", 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5063e-05 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666515+0000", 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672509+0000", 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059937419999999998 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672538+0000", 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9645000000000002e-05 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189946 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.113198385, 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068095500000000002, 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.850 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189946 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673035+0000", 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673044+0000", 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673060+0000", 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5872999999999999e-05 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673068+0000", 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1759999999999993e-06 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673112+0000", 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3998000000000001e-05 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673189+0000", 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6330999999999998e-05 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673305+0000", 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011642 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.678066+0000", 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047612519999999997 2024-09-13T09:29:38.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679714+0000", 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001647665 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679755+0000", 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0941e-05 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679846+0000", 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1558000000000007e-05 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190167 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.512510978999998, 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024116239999999998, 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190167 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.856 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273724+0000", 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273722+0000", 2024-09-13T09:29:38.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273736+0000", 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.345e-05 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273767+0000", 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1359000000000002e-05 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273783+0000", 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5549999999999999e-05 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.273893+0000", 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110264 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274045+0000", 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015171299999999999 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.274391+0000", 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034663899999999999 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.275713+0000", 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013217630000000001 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276007+0000", 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029441100000000001 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276074+0000", 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7124000000000003e-05 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.276136+0000", 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1420999999999994e-05 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190169 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:38.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.508124397, 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029159210000000001, 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190169 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278109+0000", 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278124+0000", 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5449000000000001e-05 2024-09-13T09:29:38.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278157+0000", 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2602999999999999e-05 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278178+0000", 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1195999999999998e-05 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278298+0000", 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-13T09:29:38.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278456+0000", 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015802899999999999 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278817+0000", 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036107899999999999 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280193+0000", 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001375392 2024-09-13T09:29:38.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280781+0000", 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058817699999999995 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280865+0000", 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3697999999999997e-05 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281027+0000", 2024-09-13T09:29:38.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016203300000000001 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190170 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.505106605000002, 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023661509999999999, 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190170 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281129+0000", 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281127+0000", 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281140+0000", 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3482999999999999e-05 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281164+0000", 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4369999999999999e-05 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281182+0000", 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8016000000000002e-05 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281324+0000", 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014187999999999999 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281446+0000", 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121198 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:38.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281690+0000", 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024418399999999999 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283024+0000", 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013344030000000001 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283315+0000", 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029132399999999998 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283392+0000", 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6452000000000006e-05 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.283495+0000", 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010284400000000001 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190173 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:38.872 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.501197188999999, 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025929310000000001, 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190173 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285038+0000", 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285036+0000", 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285047+0000", 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1083e-05 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285070+0000", 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3050000000000001e-05 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285082+0000", 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1749999999999999e-05 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285169+0000", 2024-09-13T09:29:38.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6514000000000006e-05 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285332+0000", 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016333 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.285618+0000", 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028579000000000001 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.286946+0000", 2024-09-13T09:29:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001327679 2024-09-13T09:29:38.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287517+0000", 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00057192199999999999 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287571+0000", 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3903000000000002e-05 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.287631+0000", 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.961e-05 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:190198 3.11 e198/197 3:8daa9abf:::rbd_data.11189888a71f.0000000000000001:head v 198'880, mlcod=193'877)", 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.468874364000001, 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023753540000000001, 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:38.879 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317361+0000", 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:38.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317359+0000", 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317375+0000", 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6348999999999999e-05 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317416+0000", 2024-09-13T09:29:38.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0562000000000002e-05 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317432+0000", 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5957e-05 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317658+0000", 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000225768 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:38.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.317716+0000", 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7819000000000003e-05 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319644+0000", 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019285999999999999 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.319736+0000", 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2034999999999996e-05 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:38.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:38.884 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:38.885 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:38.886 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:38.882+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:38.986 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:38.984+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:39.004 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:39.005 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:39.005 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.015 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:39.087 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:39.082+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:39.122 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.122 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:39.123 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:39.123 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.132 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:39.188 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:39.184+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:39.243 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:39.243 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:46792 3.1b 3:dc89be4e:::rbd_object_map.111838d2a860:head [read 0~43] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 177.67728323, 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0112884499999999, 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.244 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 46792 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562093+0000", 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.245 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562091+0000", 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562097+0000", 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562106+0000", 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1090000000000004e-06 2024-09-13T09:29:39.246 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.562114+0000", 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8970000000000008e-06 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563253+0000", 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0011395400000000001 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:39.247 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:41.563263+0000", 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8819999999999996e-06 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572444+0000", 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009180714 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.572552+0000", 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000108245 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.248 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:26:42.573381+0000", 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00082896900000000004 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:126493 3.12 3.16fd5272 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 145.089624851, 2024-09-13T09:29:39.249 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68124472700000005, 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "reached pg", 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 126493 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.250 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149751+0000", 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149750+0000", 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149756+0000", 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3299999999999998e-06 2024-09-13T09:29:39.251 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149769+0000", 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2767e-05 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.252 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149774+0000", 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8830000000000003e-06 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149818+0000", 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3822999999999997e-05 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.149826+0000", 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0879999999999995e-06 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.253 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830963+0000", 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.68113636 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:14.830996+0000", 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3294999999999999e-05 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:171006 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 31086592~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.254 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.959332726, 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.055737449000000001, 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 171006 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.255 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280043+0000", 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.256 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280046+0000", 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7499999999999999e-06 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280052+0000", 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9490000000000001e-06 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.280057+0000", 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:39.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315459+0000", 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.035402664 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.315605+0000", 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000145566 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.258 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.335781+0000", 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.020175986 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:178896 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 739328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 92.375122274999995, 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049352241999999998, 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.259 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 178896 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864254+0000", 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.260 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864253+0000", 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864315+0000", 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2162999999999994e-05 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864325+0000", 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7550000000000006e-06 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.261 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.864332+0000", 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.907949+0000", 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.043617300999999997 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.908030+0000", 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1267999999999995e-05 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.262 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.913606+0000", 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0055758700000000001 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182055 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1607680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973788643000006, 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048659802000000002, 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.263 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182055 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265587+0000", 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265590+0000", 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.858e-06 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265596+0000", 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:29:39.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265601+0000", 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314128+0000", 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048526897999999999 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.266 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314178+0000", 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0627999999999997e-05 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314247+0000", 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8928000000000001e-05 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182056 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1462272~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.267 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973770634999994, 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048755975, 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182056 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.268 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265605+0000", 2024-09-13T09:29:39.269 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265608+0000", 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0010000000000002e-06 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265614+0000", 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.270 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265619+0000", 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4270000000000001e-06 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314261+0000", 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048642890000000001 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314303+0000", 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1063000000000003e-05 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.271 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314361+0000", 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8913999999999999e-05 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182057 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1482752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973753295999998, 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048883043000000001, 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.272 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182057 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.273 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265623+0000", 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265622+0000", 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265625+0000", 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6690000000000002e-06 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.274 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265631+0000", 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7150000000000003e-06 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265636+0000", 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6940000000000003e-06 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314384+0000", 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048748543999999998 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.275 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314434+0000", 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9842000000000002e-05 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314506+0000", 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1882000000000001e-05 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.276 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182058 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1503232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973736039000002, 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048986676, 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182058 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.277 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265640+0000", 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.278 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265643+0000", 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.954e-06 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265648+0000", 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7379999999999999e-06 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265653+0000", 2024-09-13T09:29:39.279 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480000000000002e-06 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314521+0000", 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868341000000003 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314560+0000", 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.8998000000000002e-05 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.280 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314627+0000", 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6320000000000002e-05 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182059 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1523712~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973718503000001, 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049114968000000002, 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.281 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182059 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265658+0000", 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.282 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265657+0000", 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265660+0000", 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265666+0000", 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6389999999999997e-06 2024-09-13T09:29:39.283 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265670+0000", 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6090000000000003e-06 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314649+0000", 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048978890999999997 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314698+0000", 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9288e-05 2024-09-13T09:29:39.284 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314773+0000", 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4191999999999997e-05 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182060 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1544192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973701433000002, 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049219328, 2024-09-13T09:29:39.285 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182060 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.286 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265675+0000", 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265674+0000", 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265677+0000", 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5270000000000001e-06 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.287 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265683+0000", 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265686+0000", 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1080000000000001e-06 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314788+0000", 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049102163999999997 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.288 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314829+0000", 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1247e-05 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314894+0000", 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4684000000000006e-05 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.289 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182061 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1628160~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973687773999998, 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049348507999999999, 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182061 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:39.290 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265688+0000", 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.291 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265690+0000", 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9939999999999999e-06 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265693+0000", 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1609999999999999e-06 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265696+0000", 2024-09-13T09:29:39.292 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314917+0000", 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049221574999999997 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.314970+0000", 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2215000000000001e-05 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.293 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315037+0000", 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7176999999999997e-05 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182062 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1564672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973678101000004, 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049452410000000002, 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.294 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182062 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:39.295 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265698+0000", 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265699+0000", 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4750000000000001e-06 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.296 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265704+0000", 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6959999999999996e-06 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265708+0000", 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315051+0000", 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049342219 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.297 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315090+0000", 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9161e-05 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315150+0000", 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0708999999999998e-05 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.298 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182063 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1648640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973663556000005, 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049577870000000003, 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182063 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:39.299 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265713+0000", 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265712+0000", 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.300 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265715+0000", 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5739999999999998e-06 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265720+0000", 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4439999999999999e-06 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265725+0000", 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.301 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315173+0000", 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049448331999999998 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315224+0000", 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0448999999999998e-05 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315290+0000", 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6536000000000001e-05 2024-09-13T09:29:39.302 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182064 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1585152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973646888999994, 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049674550999999997, 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.303 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182064 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.304 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265729+0000", 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265731+0000", 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6070000000000002e-06 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265738+0000", 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0399999999999998e-06 2024-09-13T09:29:39.305 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265742+0000", 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5889999999999996e-06 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315302+0000", 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049560189999999997 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315347+0000", 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4984999999999997e-05 2024-09-13T09:29:39.306 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315404+0000", 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6468999999999998e-05 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182065 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1669120~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.973629798000005, 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049810353000000002, 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.307 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182065 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:39.308 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265746+0000", 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265749+0000", 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.309 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265755+0000", 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3709999999999996e-06 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.265760+0000", 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9629999999999997e-06 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315428+0000", 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049667713000000002 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.310 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315487+0000", 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8968000000000002e-05 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.315557+0000", 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.0109000000000002e-05 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.311 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182070 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1646592~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.971774221999993, 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048654393999999997, 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.312 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182070 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267602+0000", 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.313 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267601+0000", 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267604+0000", 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.314 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267609+0000", 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3059999999999997e-06 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267614+0000", 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6269999999999999e-06 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316151+0000", 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536982999999999 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.315 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316201+0000", 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9570000000000001e-05 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316256+0000", 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5698000000000001e-05 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.316 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182071 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1730560~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.971758113999996, 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048771657000000003, 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182071 2024-09-13T09:29:39.317 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.318 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267618+0000", 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267620+0000", 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.319 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267625+0000", 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.775e-06 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267629+0000", 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.16e-06 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316279+0000", 2024-09-13T09:29:39.320 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048649924999999997 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316327+0000", 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7982999999999998e-05 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316390+0000", 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2935999999999995e-05 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182072 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1667072~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.971743750000002, 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048879971000000001, 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182072 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.322 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.323 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267632+0000", 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267634+0000", 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0269999999999998e-06 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267639+0000", 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.578e-06 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.324 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267643+0000", 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2270000000000004e-06 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316407+0000", 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048763582 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.325 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316455+0000", 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8262e-05 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316512+0000", 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7546000000000001e-05 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182073 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1751040~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:39.326 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.971729921999994, 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048989151000000002, 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182073 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.327 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267646+0000", 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267648+0000", 2024-09-13T09:29:39.328 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0229999999999999e-06 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267654+0000", 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.947e-06 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267658+0000", 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.329 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316526+0000", 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048868047999999997 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316571+0000", 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4468000000000002e-05 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316635+0000", 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.4597999999999994e-05 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.330 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:182074 3.1b 3:df82c1a3:::rbd_data.111827e6de71.0000000000000003:30 [read 1687552~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 91.971712221999994, 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049087884999999998, 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 182074 2024-09-13T09:29:39.331 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.332 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267664+0000", 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267666+0000", 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1220000000000002e-06 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267671+0000", 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.333 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.267675+0000", 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.065e-06 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316649+0000", 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048974555000000003 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316695+0000", 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5759e-05 2024-09-13T09:29:39.334 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:07.316752+0000", 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.6728000000000003e-05 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.335 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.337 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:39.337 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:39.332+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:39.438 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:39.436+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:39.453 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:39.453 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:39.453 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.463 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:39.523 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:39.569 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:39.566+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:39.570 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:39.570 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:39.571 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.581 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:39.639 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:39.634+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:39.652 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:39.652 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.689 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:39.689 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141110 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 303104~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108948192, 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048487047999999998, 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.690 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141110 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.691 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576387+0000", 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576389+0000", 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9630000000000001e-06 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576394+0000", 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1209999999999998e-06 2024-09-13T09:29:39.692 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576397+0000", 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.8990000000000001e-06 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624676+0000", 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048279117000000003 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.693 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624780+0000", 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104131 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624874+0000", 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4067000000000006e-05 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.694 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141112 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 344064~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.10893595500001, 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048612216, 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141112 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.695 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576399+0000", 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576401+0000", 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576407+0000", 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1329999999999998e-06 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576411+0000", 2024-09-13T09:29:39.697 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4660000000000001e-06 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624912+0000", 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048500697000000002 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.624962+0000", 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0630999999999997e-05 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.698 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625012+0000", 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9326000000000002e-05 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141114 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 360448~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.10892055799999, 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048836427000000002, 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.699 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141114 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.700 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576415+0000", 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576417+0000", 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1710000000000001e-06 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.701 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576422+0000", 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8919999999999999e-06 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576426+0000", 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6589999999999998e-06 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625061+0000", 2024-09-13T09:29:39.702 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048634322000000001 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625161+0000", 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010071199999999999 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625251+0000", 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9930000000000001e-05 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.703 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141116 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 385024~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108905683, 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048911740000000002, 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141116 2024-09-13T09:29:39.704 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576430+0000", 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.705 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576429+0000", 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576432+0000", 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576438+0000", 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3679999999999998e-06 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.706 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576443+0000", 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8459999999999999e-06 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625263+0000", 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048820370000000002 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625300+0000", 2024-09-13T09:29:39.707 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6943000000000002e-05 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625341+0000", 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1186999999999997e-05 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141118 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 401408~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:39.708 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108888474, 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049115942000000003, 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141118 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.709 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576447+0000", 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.710 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576453+0000", 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9209999999999997e-06 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576458+0000", 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7080000000000002e-06 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576463+0000", 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.711 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625377+0000", 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048913783000000002 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625474+0000", 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7379000000000004e-05 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625563+0000", 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8485000000000004e-05 2024-09-13T09:29:39.712 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141120 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 442368~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108868019, 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049185008000000002, 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.713 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141120 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.714 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576467+0000", 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576469+0000", 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3590000000000002e-06 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576475+0000", 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3090000000000003e-06 2024-09-13T09:29:39.715 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576481+0000", 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625576+0000", 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049095314000000001 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.716 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625611+0000", 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4708e-05 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625652+0000", 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1542000000000003e-05 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141122 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 458752~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:39.717 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108849939, 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049387957000000003, 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141122 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:39.718 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576485+0000", 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576487+0000", 2024-09-13T09:29:39.719 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0049999999999999e-06 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576493+0000", 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8379999999999998e-06 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576498+0000", 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8829999999999997e-06 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.720 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625687+0000", 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049189530000000002 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625784+0000", 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6173000000000002e-05 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625873+0000", 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9802999999999994e-05 2024-09-13T09:29:39.721 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141124 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 483328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108833563, 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049460946999999998, 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.722 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141124 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.723 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576502+0000", 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576504+0000", 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2230000000000001e-06 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576509+0000", 2024-09-13T09:29:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2959999999999998e-06 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576514+0000", 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625886+0000", 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049372164000000003 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.725 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625920+0000", 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.4008999999999999e-05 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.625963+0000", 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3003000000000001e-05 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.726 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141126 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 499712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108818212, 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049588259000000003, 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141126 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576517+0000", 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.728 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576519+0000", 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.142e-06 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576525+0000", 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8499999999999999e-06 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576530+0000", 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8690000000000003e-06 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.729 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626009+0000", 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049479391999999997 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626061+0000", 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2079000000000001e-05 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626105+0000", 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4217e-05 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.730 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141135 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 12288~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108801508, 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049772010999999998, 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141135 2024-09-13T09:29:39.731 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.732 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576534+0000", 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576536+0000", 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.221e-06 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576542+0000", 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3729999999999998e-06 2024-09-13T09:29:39.733 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576547+0000", 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8269999999999997e-06 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626141+0000", 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049594045000000003 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626220+0000", 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8684999999999996e-05 2024-09-13T09:29:39.734 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626306+0000", 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6154000000000002e-05 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141136 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 53248~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.108783426, 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049931538999999997, 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.735 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141136 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.736 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576552+0000", 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576554+0000", 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2649999999999999e-06 2024-09-13T09:29:39.737 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576560+0000", 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7259999999999999e-06 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576565+0000", 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9590000000000003e-06 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.738 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626332+0000", 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049767429000000002 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626400+0000", 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7895e-05 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626484+0000", 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.3541000000000005e-05 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.739 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141137 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 69632~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.10876668100001, 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050041013000000002, 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141137 2024-09-13T09:29:39.740 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576569+0000", 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.741 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576568+0000", 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576571+0000", 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576576+0000", 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:29:39.742 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576581+0000", 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5220000000000001e-06 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626499+0000", 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049918487999999997 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626548+0000", 2024-09-13T09:29:39.743 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8748999999999997e-05 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626610+0000", 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1901000000000002e-05 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141138 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 94208~1640] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.744 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.10872202199999, 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.050329305999999997, 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141138 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.745 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576613+0000", 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.746 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576615+0000", 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3149999999999999e-06 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576625+0000", 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5990000000000008e-06 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.747 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.576628+0000", 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.05e-06 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626626+0000", 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049998287000000002 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626669+0000", 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2465000000000001e-05 2024-09-13T09:29:39.748 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.626943+0000", 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00027391900000000002 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141146 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 290816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:39.749 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.105772582, 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048577502000000002, 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141146 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:39.750 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579563+0000", 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.751 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579565+0000", 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7439999999999999e-06 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579576+0000", 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0487e-05 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579581+0000", 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7570000000000004e-06 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.752 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628045+0000", 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048464759000000003 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628093+0000", 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7295e-05 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628140+0000", 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7673999999999998e-05 2024-09-13T09:29:39.753 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141147 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 307200~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.105752044, 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048686826000000002, 2024-09-13T09:29:39.754 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141147 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.755 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579583+0000", 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579585+0000", 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:29:39.756 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579596+0000", 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0577000000000001e-05 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579601+0000", 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.757 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628171+0000", 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048569067 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628223+0000", 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2753000000000001e-05 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628270+0000", 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6937000000000001e-05 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.758 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141148 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 348160~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.10572942899999, 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048793536999999998, 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141148 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.759 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579606+0000", 2024-09-13T09:29:39.760 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579609+0000", 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.761 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579615+0000", 2024-09-13T09:29:39.764 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579620+0000", 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0629999999999996e-06 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628304+0000", 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048683931999999999 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.765 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628350+0000", 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6127000000000001e-05 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628400+0000", 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9282e-05 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.766 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141149 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 364544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.105710561, 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048910767000000001, 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141149 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.767 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579625+0000", 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.768 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579627+0000", 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.046e-06 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579630+0000", 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.3349999999999998e-06 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579635+0000", 2024-09-13T09:29:39.769 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.622e-06 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628443+0000", 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048808011999999998 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.770 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628489+0000", 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6137000000000002e-05 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628536+0000", 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6906000000000001e-05 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141150 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 389120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.771 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.105675967, 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.049020899, 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141150 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.772 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579659+0000", 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.773 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579662+0000", 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.9119999999999998e-06 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579668+0000", 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.1800000000000001e-06 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.579675+0000", 2024-09-13T09:29:39.774 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1380000000000001e-06 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628567+0000", 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048892061000000001 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628621+0000", 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.4131000000000001e-05 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.775 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.628680+0000", 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9055999999999998e-05 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141190 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 350208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.101397062, 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048474977000000002, 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.776 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141190 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.777 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583938+0000", 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583941+0000", 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.756e-06 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.778 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583948+0000", 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8120000000000002e-06 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583952+0000", 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8450000000000002e-06 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632330+0000", 2024-09-13T09:29:39.779 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048377156999999997 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632372+0000", 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.2094000000000001e-05 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632413+0000", 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1624e-05 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.780 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:141191 3.19 3:9f3df048:::rbd_data.1118fafd09f3.000000000000010f:2a [read 366592~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e114)", 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 112.10133309, 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048536852999999998, 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 141191 2024-09-13T09:29:39.781 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584002+0000", 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:39.782 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.583957+0000", 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999542 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584006+0000", 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9181999999999998e-05 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584017+0000", 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0849999999999999e-05 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.783 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.584023+0000", 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.387e-06 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632447+0000", 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.048424277000000002 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632491+0000", 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:29:39.784 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:47.632539+0000", 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.7985000000000002e-05 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:39.785 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:39.785 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:29:39.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:39.788 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:39.782+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:39.888 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:39.884+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:39.903 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:39.904 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:39.904 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:39.904 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:39.914 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:39.989 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:39.986+0000 7f78c3db8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43757) UID: 0 2024-09-13T09:29:40.022 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:40.022 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:40.022 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:40.022 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:40.022 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:40.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:40.090 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:40.088+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:40.142 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:40.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.714108472000007, 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:40.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.714093415999997, 2024-09-13T09:29:40.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:40.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.151 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.714084008, 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.152 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:40.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.156 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.714069714999994, 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:40.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713984009000001, 2024-09-13T09:29:40.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:40.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:40.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713627759999994, 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:40.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:40.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713580923999999, 2024-09-13T09:29:40.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:40.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:40.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713541938000006, 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:40.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713521794000002, 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713502726000002, 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713460636999997, 2024-09-13T09:29:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713347135000006, 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:40.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:40.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:40.200 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713328955999998, 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713319210999998, 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:40.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713309023999997, 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.210 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:40.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.214 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713216157999994, 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:40.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:40.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:40.219 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.713112745999993, 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.712790867999999, 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:40.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:40.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.712374926999999, 2024-09-13T09:29:40.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:40.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.712121217000004, 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:40.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.237 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:40.239 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is set 2024-09-13T09:29:40.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:40.240 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:40.234+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:40.339 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:40.334+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:40.398 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:40.398 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:40.398 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:40.398 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:40.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:40.439 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:29:40.441 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:40.436+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:40.511 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:40.511 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:40.511 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:40.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:40.542 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:40.538+0000 7f78c3db8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43757) UID: 0 2024-09-13T09:29:40.594 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:40.617 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:40.636 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:40.643 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:40.638+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:40.658 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:40.672 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 237.70804148900001, 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 174.59482174499999, 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 174.59442372999999, 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.91931685399999, 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:40.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.91930192199999, 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:40.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:40.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:40.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:40.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:40.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:40.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:40.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.919268816, 2024-09-13T09:29:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.91925252300001, 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.91923640799999, 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.91920536699999, 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.91917374400001, 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.906225683, 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.90620862700001, 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.90619245900001, 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.90597879699999, 2024-09-13T09:29:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.905959486, 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.90594309299999, 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.90592743100001, 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 173.90590107700001, 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 127.37738835, 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.265474235, 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:40.775 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:40.776 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:40.776 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:40.777 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:40.777 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:40.778 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:40.778 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:40.772+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:40.779 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:40.779 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:40.877 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:40.874+0000 7f78c3db8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43757) UID: 0 2024-09-13T09:29:40.937 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:40.937 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:40.948 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:40.993 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:40.990+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:41.053 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.053 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.063 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:41.078 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:41.074+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:41.173 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.174 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.179 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:41.174+0000 7f78c3db8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43757) UID: 0 2024-09-13T09:29:41.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:41.280 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:41.276+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:41.298 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.298 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:41.381 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:41.378+0000 7f78c3db8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43757) UID: 0 2024-09-13T09:29:41.415 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.415 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:41.482 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:41.480+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:41.530 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.530 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:41.583 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:41.578+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:41.657 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.658 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.668 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:29:41.709 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:41.706+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:41.784 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:41.780+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:41.824 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.824 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.834 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:41.885 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:41.884+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:41.945 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:41.945 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:41.955 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:29:41.986 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-13T09:29:41.984+0000 7fd01e1f1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 22313) UID: 0 2024-09-13T09:29:42.061 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.061 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:29:42.087 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:42.084+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:42.178 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.178 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.188 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:29:42.189 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:42.186+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:42.241 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:29:42.289 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:42.286+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:42.336 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.337 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.350 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:29:42.390 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:42.388+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:42.457 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.457 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.467 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:29:42.491 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:42.486+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:42.574 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.575 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.585 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:29:42.629 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-09-13T09:29:42.626+0000 7ff6df423700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33201) UID: 0 2024-09-13T09:29:42.693 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:42.692+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:42.738 INFO:teuthology.orchestra.run.smithi063.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.739 INFO:teuthology.orchestra.run.smithi063.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.750 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:29:42.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:29:42.794 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:42.790+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:42.866 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.866 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:29:42.894 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:42.890+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:42.929 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0: {} 2024-09-13T09:29:42.953 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1: {} 2024-09-13T09:29:42.971 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2: {} 2024-09-13T09:29:42.984 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-13T09:29:42.984 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:29:42.990 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3: {} 2024-09-13T09:29:42.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:29:42.995 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:42.990+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:43.014 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4: {} 2024-09-13T09:29:43.038 INFO:teuthology.orchestra.run.smithi063.stdout:osd.6: {} 2024-09-13T09:29:43.061 INFO:teuthology.orchestra.run.smithi063.stdout:osd.7: {} 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:29:43.062 INFO:teuthology.orchestra.run.smithi063.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:29:43.066 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:29:43.067 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:29:43.096 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:43.092+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:43.101 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173406 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2660352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673528665000006, 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059459261999999999, 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.102 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173406 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424587+0000", 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424586+0000", 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424589+0000", 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424596+0000", 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424599+0000", 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3079999999999999e-06 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483924+0000", 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059325510999999997 2024-09-13T09:29:43.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.483974+0000", 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9295999999999997e-05 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484046+0000", 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2343999999999997e-05 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173407 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2680832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673513608999997, 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059714567000000003, 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173407 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424602+0000", 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424603+0000", 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.846e-06 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424606+0000", 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8669999999999998e-06 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424609+0000", 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-13T09:29:43.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484083+0000", 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059474314 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484190+0000", 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106476 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484316+0000", 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012646200000000001 2024-09-13T09:29:43.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173408 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2701312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673504201, 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059866638999999999, 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173408 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424611+0000", 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424612+0000", 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.333e-06 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424615+0000", 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7769999999999998e-06 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424618+0000", 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8210000000000002e-06 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484354+0000", 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059735842999999997 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484406+0000", 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2391e-05 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484478+0000", 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1680999999999994e-05 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173409 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2721792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673489907999993, 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060139330999999997, 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173409 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424626+0000", 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424625+0000", 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424627+0000", 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.968e-06 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424633+0000", 2024-09-13T09:29:43.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424638+0000", 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.583e-06 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484519+0000", 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059880883000000003 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484628+0000", 2024-09-13T09:29:43.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109053 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484765+0000", 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000137133 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173411 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673404202, 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060208484999999999, 2024-09-13T09:29:43.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173411 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424711+0000", 2024-09-13T09:29:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424710+0000", 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424718+0000", 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3139999999999998e-06 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424727+0000", 2024-09-13T09:29:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.424733+0000", 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.925e-06 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484788+0000", 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060054519000000001 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484838+0000", 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0723000000000002e-05 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484920+0000", 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1225999999999998e-05 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173413 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673047952999994, 2024-09-13T09:29:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060136740000000001, 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173413 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425067+0000", 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425066+0000", 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425073+0000", 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1500000000000004e-06 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425082+0000", 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3179999999999996e-06 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425098+0000", 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5693999999999999e-05 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.484957+0000", 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.059859950000000002 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485068+0000", 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011089799999999999 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485204+0000", 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135845 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173415 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2783232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.673001116999998, 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060238055999999998, 2024-09-13T09:29:43.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173415 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425114+0000", 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425117+0000", 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6610000000000002e-06 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425132+0000", 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4972e-05 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425146+0000", 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4012999999999999e-05 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485229+0000", 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060082125 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485276+0000", 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7664999999999998e-05 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485352+0000", 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6143000000000006e-05 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.131 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173416 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2803712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672962131000006, 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060347400000000002, 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173416 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425153+0000", 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425156+0000", 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560000000000001e-06 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425164+0000", 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.396e-06 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425169+0000", 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0640000000000001e-06 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485383+0000", 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060214018000000001 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485433+0000", 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0299000000000001e-05 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485501+0000", 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7720999999999998e-05 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173418 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2824192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672941987000002, 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060469651999999999, 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173418 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.136 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425173+0000", 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425176+0000", 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0709999999999999e-06 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425182+0000", 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.507e-06 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425186+0000", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485526+0000", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060339476000000003 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485574+0000", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7817e-05 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485643+0000", 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9333999999999997e-05 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173419 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672922919000001, 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060675044999999997, 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173419 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425192+0000", 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425205+0000", 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3176000000000001e-05 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425227+0000", 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1124999999999998e-05 2024-09-13T09:29:43.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425231+0000", 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2010000000000001e-06 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485667+0000", 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060436770000000001 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485752+0000", 2024-09-13T09:29:43.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4466000000000001e-05 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485868+0000", 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115579 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173421 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2865152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672880829999997, 2024-09-13T09:29:43.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060794782999999998, 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173421 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425235+0000", 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425234+0000", 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425237+0000", 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425246+0000", 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3659999999999998e-06 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425264+0000", 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8417e-05 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485904+0000", 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060640055999999998 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.485965+0000", 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0362999999999998e-05 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486029+0000", 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173423 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2885632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672767328000006, 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060829633000000001, 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173423 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425348+0000", 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425347+0000", 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425351+0000", 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2309999999999998e-06 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425359+0000", 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8739999999999995e-06 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425363+0000", 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6550000000000002e-06 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486065+0000", 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060702327 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486116+0000", 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1096e-05 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486178+0000", 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173424 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2906112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.152 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672749148999998, 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060954761000000003, 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173424 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425366+0000", 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425368+0000", 2024-09-13T09:29:43.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0109999999999999e-06 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425371+0000", 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425374+0000", 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.711e-06 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486212+0000", 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060838849 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486256+0000", 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4023999999999997e-05 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486321+0000", 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4533e-05 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173426 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672739403999998, 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061108219999999998, 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173426 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.157 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425376+0000", 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:43.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425377+0000", 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4389999999999999e-06 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425381+0000", 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425384+0000", 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8080000000000001e-06 2024-09-13T09:29:43.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486355+0000", 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060971157999999998 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486427+0000", 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1709000000000001e-05 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486484+0000", 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7497999999999997e-05 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173428 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2947072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672729216999997, 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061229577, 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.161 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173428 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425386+0000", 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425387+0000", 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.271e-06 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425392+0000", 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7160000000000002e-06 2024-09-13T09:29:43.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425396+0000", 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.126e-06 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486514+0000", 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061118366 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486560+0000", 2024-09-13T09:29:43.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5586e-05 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486616+0000", 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5724999999999999e-05 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173430 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2967552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672636350999994, 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061264441000000003, 2024-09-13T09:29:43.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173430 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425479+0000", 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425478+0000", 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425486+0000", 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3420000000000002e-06 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425500+0000", 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4249e-05 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425507+0000", 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2250000000000003e-06 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486643+0000", 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061135592000000002 2024-09-13T09:29:43.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486686+0000", 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3163e-05 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486744+0000", 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494000000000002e-05 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173432 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 2988032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672532939000007, 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061292625000000003, 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173432 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425582+0000", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425589+0000", 2024-09-13T09:29:43.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7109999999999995e-06 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425607+0000", 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7340999999999999e-05 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425620+0000", 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3468999999999999e-05 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486776+0000", 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061155552000000002 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486825+0000", 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9626999999999998e-05 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486875+0000", 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9829e-05 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173434 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3008512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.672211060999999, 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.061096487999999997, 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173434 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425904+0000", 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425688+0000", 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999783 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425914+0000", 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022618000000000001 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425921+0000", 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1940000000000001e-06 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.425999+0000", 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8007e-05 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486906+0000", 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060906801000000003 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.486948+0000", 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2347e-05 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487001+0000", 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2747000000000002e-05 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173436 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3028992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.671795119999999, 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060819581999999997, 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173436 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426320+0000", 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426017+0000", 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9996963 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426332+0000", 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031546299999999998 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426348+0000", 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5733999999999999e-05 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426355+0000", 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0979999999999996e-06 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487034+0000", 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060678957999999998 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487076+0000", 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1952000000000001e-05 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487140+0000", 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4035999999999995e-05 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:173438 3.a 3:512f8bd6:::rbd_data.111827e6de71.0000000000000001:30 [read 3049472~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.671541410000003, 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060714130999999998, 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 173438 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.182 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426574+0000", 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426361+0000", 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997873 2024-09-13T09:29:43.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426579+0000", 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000218106 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426587+0000", 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8450000000000003e-06 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.426603+0000", 2024-09-13T09:29:43.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5461999999999999e-05 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487172+0000", 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.060569919 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487225+0000", 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2633999999999998e-05 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:28:06.487288+0000", 2024-09-13T09:29:43.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3015999999999994e-05 2024-09-13T09:29:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.186 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:43.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:29:43.229 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is set 2024-09-13T09:29:43.233 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:43.232+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:43.299 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:43.296+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:43.304 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:43.304 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:43.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:29:43.399 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-13T09:29:43.396+0000 7f3dc9b06700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31826) UID: 0 2024-09-13T09:29:43.420 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:43.421 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:43.421 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:43.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:29:43.500 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-09-13T09:29:43.498+0000 7f78c3db8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43757) UID: 0 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4211.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:43.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 240.62063599300001, 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43182651999999999, 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4211", 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/621325887", 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:43.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961015+0000", 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961013+0000", 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961021+0000", 2024-09-13T09:29:43.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3620000000000006e-06 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961033+0000", 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961045+0000", 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1619e-05 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961125+0000", 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0438000000000005e-05 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:42.961135+0000", 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.764e-06 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392624+0000", 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.43148925799999999 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:25:43.392841+0000", 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000216825 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76886 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1567744~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 177.50741624899999, 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055231497999999997, 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.590 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76886 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074234+0000", 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074237+0000", 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074242+0000", 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7470000000000002e-06 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074247+0000", 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129241+0000", 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054994375999999998 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129360+0000", 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119188 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129466+0000", 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105629 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.594 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:76941 3.d 3:b5888148:::rbd_data.111838d2a860.0000000000000002:17 [read 1913856~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 177.50701823399999, 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054603394999999999, 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 76941 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074632+0000", 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074635+0000", 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.588e-06 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074640+0000", 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7490000000000004e-06 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.074645+0000", 2024-09-13T09:29:43.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129053+0000", 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054408297000000001 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129131+0000", 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8510999999999994e-05 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.129236+0000", 2024-09-13T09:29:43.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010438300000000001 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86505 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.83191135800001, 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054610849000000003, 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86505 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749739+0000", 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749741+0000", 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.227e-06 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749746+0000", 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.95e-06 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749750+0000", 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.929e-06 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804173+0000", 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054422909999999998 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804249+0000", 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6131999999999995e-05 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804350+0000", 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101022 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.603 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86506 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 958464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.83189642599999, 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054792050000000002, 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86506 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749754+0000", 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749756+0000", 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5220000000000002e-06 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749762+0000", 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7679999999999997e-06 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749766+0000", 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2180000000000001e-06 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804376+0000", 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.05460918 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804449+0000", 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3761000000000001e-05 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804546+0000", 2024-09-13T09:29:43.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6980999999999999e-05 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86508 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 973824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.83186332, 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055043531999999999, 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86508 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749787+0000", 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749789+0000", 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3379999999999999e-06 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749795+0000", 2024-09-13T09:29:43.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5470000000000004e-06 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749800+0000", 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6890000000000003e-06 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804576+0000", 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054776460999999999 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804681+0000", 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010488399999999999 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804831+0000", 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014998600000000001 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86509 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.83184702700001, 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055280092000000003, 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86509 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749804+0000", 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749803+0000", 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749806+0000", 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2069999999999998e-06 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749811+0000", 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5929999999999996e-06 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749815+0000", 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2570000000000001e-06 2024-09-13T09:29:43.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804879+0000", 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055064106000000002 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.804968+0000", 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8233999999999996e-05 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805084+0000", 2024-09-13T09:29:43.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116044 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86510 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1004544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.83183091199999, 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055514919000000003, 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86510 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749820+0000", 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749819+0000", 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749822+0000", 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2910000000000002e-06 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749827+0000", 2024-09-13T09:29:43.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.254e-06 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749831+0000", 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2869999999999998e-06 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805136+0000", 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055304868 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805229+0000", 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2383e-05 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805335+0000", 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106184 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86512 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1019904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.83179987099999, 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055625213999999999, 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86512 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749851+0000", 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749850+0000", 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749852+0000", 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0090000000000002e-06 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749858+0000", 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3249999999999998e-06 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749862+0000", 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6609999999999999e-06 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805365+0000", 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055502109000000001 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805419+0000", 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4555999999999997e-05 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805476+0000", 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6898000000000003e-05 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:86514 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.831768248, 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055743481999999997, 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 86514 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749882+0000", 2024-09-13T09:29:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749889+0000", 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3549999999999999e-06 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749896+0000", 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3339999999999998e-06 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.749901+0000", 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9300000000000002e-06 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805501+0000", 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055599948000000003 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805552+0000", 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1523000000000001e-05 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.805626+0000", 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3702999999999996e-05 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87008 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 929792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.818820187, 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054579674000000002, 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87008 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762831+0000", 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762830+0000", 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762832+0000", 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.503e-06 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762838+0000", 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762842+0000", 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817240+0000", 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054397884000000001 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817314+0000", 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3573999999999997e-05 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817410+0000", 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6429999999999997e-05 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87009 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 945152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81880313100001, 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055090367000000001, 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87009 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762848+0000", 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762847+0000", 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762850+0000", 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.362e-06 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762855+0000", 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4389999999999999e-06 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762860+0000", 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6879999999999998e-06 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817447+0000", 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054587504000000002 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817799+0000", 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035170999999999998 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817938+0000", 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013902599999999999 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87011 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 960512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81878696300001, 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055308582000000002, 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87011 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762864+0000", 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762863+0000", 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.762876+0000", 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763063+0000", 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018656200000000001 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763071+0000", 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.909e-06 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.817970+0000", 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.054899099999999999 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818059+0000", 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9382000000000006e-05 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818172+0000", 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000113188 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87013 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 975872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81857330099999, 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055296807000000003, 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87013 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763077+0000", 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763081+0000", 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7570000000000002e-06 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763088+0000", 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0129999999999996e-06 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763092+0000", 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6750000000000001e-06 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818211+0000", 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055118549000000003 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818277+0000", 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6625999999999995e-05 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818374+0000", 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6768999999999995e-05 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87015 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 991232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81855399, 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055491632999999999, 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87015 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763097+0000", 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763096+0000", 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763099+0000", 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763104+0000", 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5119999999999999e-06 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763109+0000", 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.459e-06 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818404+0000", 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055295506000000001 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818481+0000", 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7074000000000001e-05 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818588+0000", 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000106984 2024-09-13T09:29:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87017 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1006592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81853759699999, 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055757489, 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87017 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763113+0000", 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763115+0000", 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763120+0000", 2024-09-13T09:29:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763125+0000", 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3549999999999998e-06 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818685+0000", 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055560475999999998 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818759+0000", 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3539000000000002e-05 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818871+0000", 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111829 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87019 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1021952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81852193500001, 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056000287000000003, 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87019 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763129+0000", 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763128+0000", 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763131+0000", 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2630000000000002e-06 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763136+0000", 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2170000000000002e-06 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763141+0000", 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.651e-06 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.818912+0000", 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055771813000000003 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819003+0000", 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0997999999999998e-05 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819129+0000", 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012568099999999999 2024-09-13T09:29:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:87021 3.15 3:aef46e77:::rbd_data.111838d2a860.0000000000000008:17 [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63)", 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 176.81849558100001, 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.056211721999999999, 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 87021 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763155+0000", 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763158+0000", 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0309999999999999e-06 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763164+0000", 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1369999999999999e-06 2024-09-13T09:29:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.763168+0000", 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819163+0000", 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.055995075999999998 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819247+0000", 2024-09-13T09:29:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3175000000000002e-05 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:26:46.819367+0000", 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012025 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133073 3.13 3:c9f831d8:::rbd_object_map.11182b78662d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 130.28998285399999, 2024-09-13T09:29:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0446139630000002, 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133073 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291668+0000", 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291666+0000", 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291677+0000", 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1218e-05 2024-09-13T09:29:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291694+0000", 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6983999999999999e-05 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291701+0000", 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291811+0000", 2024-09-13T09:29:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010950699999999999 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:33.291827+0000", 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5885000000000001e-05 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285694+0000", 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.99386730599999995 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:34.285709+0000", 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4591e-05 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332185+0000", 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.046476943 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332356+0000", 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000170104 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-13T09:29:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.332800+0000", 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000444482 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.334067+0000", 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001267221 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336113+0000", 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020459380000000002 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336184+0000", 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0338000000000003e-05 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:35.336282+0000", 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8224e-05 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:133448 3.13 3:c8591c93:::rbd_object_map.1118de6ae5af.0000000000000025:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 125.178068739, 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95431959300000002, 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 133448 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403582+0000", 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403581+0000", 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403589+0000", 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3459999999999994e-06 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403606+0000", 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7048e-05 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403616+0000", 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6600000000000007e-06 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403683+0000", 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7495000000000003e-05 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:38.403695+0000", 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1984000000000001e-05 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357509+0000", 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.95381398500000003 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357674+0000", 2024-09-13T09:29:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016463099999999999 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:27:39.357902+0000", 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022769000000000001 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:43.677 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:43.678 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:29:43.679 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:43.674+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:43.779 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:43.774+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:43.796 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:43.796 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:43.797 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:43.797 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:43.807 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:29:43.880 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-13T09:29:43.876+0000 7fe64b648700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 22317) UID: 0 2024-09-13T09:29:43.959 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:43.959 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:43.959 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:43.959 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:43.960 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:43.969 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:29:43.981 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-09-13T09:29:43.978+0000 7f42bd6fd700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 22200) UID: 0 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:124353 3.7 3:e2576cde:::rbd_object_map.1118fc9bb66a.000000000000001e:head [writefull 0~68] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:44.081 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 159.339146766, 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.009912567, 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 124353 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737654+0000", 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737651+0000", 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737661+0000", 2024-09-13T09:29:44.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0066e-05 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737675+0000", 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3696999999999999e-05 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737688+0000", 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3318e-05 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737932+0000", 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00024383099999999999 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for peered", 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:04.737945+0000", 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3050999999999999e-05 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741510+0000", 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.00356478 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741713+0000", 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000203191 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for subops from 7", 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.741846+0000", 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000133144 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.743592+0000", 2024-09-13T09:29:44.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001745744 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747490+0000", 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003897728 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747538+0000", 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.8733999999999998e-05 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.087 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:27:05.747566+0000", 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7749000000000001e-05 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167677 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3995648~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.096974015000001, 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060605803999999999, 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.088 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167677 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979826+0000", 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979828+0000", 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979840+0000", 2024-09-13T09:29:44.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1043000000000001e-05 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979845+0000", 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.3530000000000002e-06 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040312+0000", 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060467576000000002 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040371+0000", 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8984999999999999e-05 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040432+0000", 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0763000000000001e-05 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.092 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167678 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4016128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.096950800000002, 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06087273, 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167678 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979850+0000", 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979849+0000", 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979852+0000", 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.5730000000000002e-06 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979862+0000", 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0394000000000001e-05 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.979868+0000", 2024-09-13T09:29:44.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.835e-06 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040476+0000", 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060607815000000002 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040594+0000", 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000117644 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.040722+0000", 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000128789 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167744 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3280896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.092822483000006, 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060711318, 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.097 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167744 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983978+0000", 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983980+0000", 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1739999999999999e-06 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983984+0000", 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.983988+0000", 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.5870000000000003e-06 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044445+0000", 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060456382000000003 2024-09-13T09:29:44.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044565+0000", 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120357 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044689+0000", 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012405700000000001 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167745 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3301376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.092769923000006, 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060880933999999998, 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167745 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984030+0000", 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984029+0000", 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:44.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984035+0000", 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.0510000000000002e-06 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984052+0000", 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6937999999999999e-05 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984060+0000", 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.2369999999999992e-06 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044713+0000", 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060652162000000003 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044808+0000", 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.4967000000000001e-05 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.105 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044911+0000", 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000103774 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167746 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3321856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.092733228, 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061071983000000003, 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167746 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984067+0000", 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984066+0000", 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984070+0000", 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984077+0000", 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.562e-06 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984082+0000", 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.1710000000000001e-06 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.044945+0000", 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060862764 2024-09-13T09:29:44.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045038+0000", 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3925000000000006e-05 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045139+0000", 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100667 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167747 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.092713598000003, 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061357643000000003, 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167747 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984087+0000", 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984086+0000", 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984093+0000", 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.6560000000000003e-06 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984099+0000", 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984104+0000", 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045184+0000", 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061079646000000001 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045302+0000", 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118558 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045444+0000", 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00014200399999999999 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167748 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3362816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.092691419000005, 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061553651000000001, 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167748 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:44.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984109+0000", 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984111+0000", 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2419999999999999e-06 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984117+0000", 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.9139999999999996e-06 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.984122+0000", 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0010000000000001e-06 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045470+0000", 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061348644000000001 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045562+0000", 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1255999999999995e-05 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.045663+0000", 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100906 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167759 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3588096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.119 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.088935636000002, 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060572028, 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167759 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987865+0000", 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987864+0000", 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987866+0000", 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.897e-06 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987881+0000", 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4649e-05 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987886+0000", 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048202+0000", 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060316349999999998 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048308+0000", 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010601 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048437+0000", 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012840400000000001 2024-09-13T09:29:44.123 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167760 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3608576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.088910302000002, 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060754846000000001, 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.124 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167760 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987890+0000", 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987892+0000", 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.0360000000000001e-06 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987901+0000", 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3409999999999992e-06 2024-09-13T09:29:44.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987905+0000", 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6500000000000002e-06 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048461+0000", 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060555984 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048548+0000", 2024-09-13T09:29:44.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.6859000000000003e-05 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048645+0000", 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7314999999999999e-05 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167761 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3629056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:44.128 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.088892465000001, 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061002501000000001, 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167761 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987908+0000", 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987909+0000", 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8870000000000001e-06 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987914+0000", 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1590000000000003e-06 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987917+0000", 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.422e-06 2024-09-13T09:29:44.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048687+0000", 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060770019000000002 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048794+0000", 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000106636 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048910+0000", 2024-09-13T09:29:44.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116694 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167762 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3649536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.088880079999996, 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061205495999999998, 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.133 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167762 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987920+0000", 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987922+0000", 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9549999999999999e-06 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987929+0000", 2024-09-13T09:29:44.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3000000000000004e-06 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987933+0000", 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.6569999999999999e-06 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.048935+0000", 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061001697000000001 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049025+0000", 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0422999999999998e-05 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049126+0000", 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100809 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167763 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3670016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:44.137 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.088851319, 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06145461, 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167763 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987949+0000", 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987951+0000", 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:29:44.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987955+0000", 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.987959+0000", 2024-09-13T09:29:44.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7450000000000001e-06 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049167+0000", 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061207394999999998 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049280+0000", 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113614 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.049404+0000", 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012355600000000001 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167776 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3936256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.084699107999995, 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060510921000000002, 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.142 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167776 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:44.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992101+0000", 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992105+0000", 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.904e-06 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992116+0000", 2024-09-13T09:29:44.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0209e-05 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992123+0000", 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1369999999999996e-06 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052432+0000", 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060308822999999998 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052530+0000", 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8811e-05 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052612+0000", 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1724999999999994e-05 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167777 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3956736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.084670819999999, 2024-09-13T09:29:44.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749154999999999, 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167777 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992130+0000", 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992129+0000", 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992133+0000", 2024-09-13T09:29:44.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.783e-06 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992142+0000", 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9949999999999994e-06 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992149+0000", 2024-09-13T09:29:44.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.2200000000000003e-06 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052676+0000", 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527413000000002 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052777+0000", 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100857 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052879+0000", 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010139700000000001 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167778 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3977216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.084644686000004, 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060949202000000001, 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.151 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167778 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992156+0000", 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992155+0000", 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992159+0000", 2024-09-13T09:29:44.153 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.428e-06 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992167+0000", 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8850000000000005e-06 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992174+0000", 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.154 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.052924+0000", 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060749535 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053004+0000", 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0122000000000003e-05 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053105+0000", 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100997 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.155 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167779 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3997696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.084620048999994, 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061175022000000003, 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.156 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167779 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.157 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992180+0000", 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992183+0000", 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.602e-06 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992202+0000", 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:44.158 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992240+0000", 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7354000000000002e-05 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053167+0000", 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060927335999999999 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.159 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053257+0000", 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0290999999999993e-05 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053355+0000", 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7955999999999995e-05 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167780 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4018176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.160 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.084537475000005, 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061312257000000002, 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167780 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:44.161 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992263+0000", 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992262+0000", 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992268+0000", 2024-09-13T09:29:44.162 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.3770000000000001e-06 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992285+0000", 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6782e-05 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992298+0000", 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.311e-05 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.163 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053398+0000", 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061099267999999998 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053477+0000", 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.9871999999999997e-05 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053575+0000", 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7763999999999994e-05 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.164 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167781 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 4038656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.084494571999997, 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.06151889, 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167781 2024-09-13T09:29:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992306+0000", 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992305+0000", 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.166 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992309+0000", 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.019e-06 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992323+0000", 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3692999999999999e-05 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.167 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.992330+0000", 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.3590000000000002e-06 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053638+0000", 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.061307409 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053729+0000", 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1318999999999998e-05 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.168 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.053825+0000", 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5692999999999993e-05 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:167843 3.8 3:160bdbed:::rbd_data.111827e6de71.0000000000000006:30 [read 3221504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131)", 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 98.080868850000002, 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060752471000000002, 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:44.169 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 167843 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995932+0000", 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:44.170 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995931+0000", 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967296 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995937+0000", 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995940+0000", 2024-09-13T09:29:44.171 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.5039999999999998e-06 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:05.995943+0000", 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056470+0000", 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.060527229000000002 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:44.172 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056576+0000", 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105894 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:28:06.056684+0000", 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010815999999999999 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:44.173 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:44.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:29:44.175 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:44.170+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:44.276 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:44.270+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:44.296 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:44.297 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:44.297 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:44.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:29:44.376 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-09-13T09:29:44.370+0000 7f4e519f8700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 22199) UID: 0 2024-09-13T09:29:44.381 INFO:teuthology.orchestra.run.smithi063.stdout:All operations completed A-OK! 2024-09-13T09:29:44.413 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-13T09:29:44.413 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-13T09:29:44.414 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:44.418 DEBUG:teuthology.parallel:result is None 2024-09-13T09:29:44.418 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T09:29:44.429 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T09:29:44.465 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T09:29:44.466 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189786 3.1 e193/191 3:832e1845:::rbd_header.1118fb4e889d:head v 193'1409, mlcod=27'1)", 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.894344539000002, 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049715810000000001, 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681938+0000", 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681937+0000", 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681955+0000", 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7588000000000002e-05 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681979+0000", 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4073000000000001e-05 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.681988+0000", 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2539999999999998e-06 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682028+0000", 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0203000000000001e-05 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.682044+0000", 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5586e-05 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686830+0000", 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047863819999999996 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.686910+0000", 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9740999999999994e-05 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4376.0:189857 3.14 e193/191 3:283508b2:::rbd_data.1118fb4e889d.0000000000000002:head v 193'607, mlcod=0'0)", 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.792280929999997, 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030577590000000002, 2024-09-13T09:29:44.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784002+0000", 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784000+0000", 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784015+0000", 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4663e-05 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784043+0000", 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8521999999999999e-05 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784056+0000", 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2591e-05 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784142+0000", 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6336999999999997e-05 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.784196+0000", 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4323999999999997e-05 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.786969+0000", 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027724210000000002 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.787060+0000", 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0841999999999995e-05 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189944 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e193)", 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.676780334999997, 2024-09-13T09:29:44.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.77023277800000001, 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189944 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899502+0000", 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899501+0000", 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899511+0000", 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9010000000000006e-06 2024-09-13T09:29:44.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899528+0000", 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7445999999999999e-05 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899545+0000", 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7268000000000001e-05 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899618+0000", 2024-09-13T09:29:44.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2534999999999996e-05 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:11.899627+0000", 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.667960+0000", 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.76833267900000002 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.668076+0000", 2024-09-13T09:29:44.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116369 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.669735+0000", 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016587539999999999 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(osd.7.0:5052 3.f e194/193 3:f5427e30:::rbd_data.11189fb5ddd6.0000000000000003:b v 194'480, mlcod=0'0)", 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:44.592 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.909841597, 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060970809999999999, 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666441+0000", 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666447+0000", 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666455+0000", 2024-09-13T09:29:44.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5490000000000001e-06 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666461+0000", 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666496+0000", 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5063e-05 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.666515+0000", 2024-09-13T09:29:44.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9028000000000001e-05 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672509+0000", 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059937419999999998 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.672538+0000", 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9645000000000002e-05 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:189946 3.1d 3:bbcae922:::rbd_object_map.111851704b2b.0000000000000009:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.903245967, 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068095500000000002, 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 189946 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.597 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673037+0000", 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673035+0000", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673044+0000", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9080000000000001e-06 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673060+0000", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5872999999999999e-05 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673068+0000", 2024-09-13T09:29:44.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1759999999999993e-06 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673112+0000", 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3998000000000001e-05 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673189+0000", 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6330999999999998e-05 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.673305+0000", 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011642 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.678066+0000", 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047612519999999997 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679714+0000", 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001647665 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679755+0000", 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0941e-05 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:12.679846+0000", 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1558000000000007e-05 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.602 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190169 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198)", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.298171978999999, 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029159210000000001, 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190169 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278111+0000", 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278109+0000", 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278124+0000", 2024-09-13T09:29:44.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5449000000000001e-05 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278157+0000", 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2602999999999999e-05 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278178+0000", 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1195999999999998e-05 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278298+0000", 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012039799999999999 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278456+0000", 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015802899999999999 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.278817+0000", 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036107899999999999 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280193+0000", 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001375392 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280781+0000", 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058817699999999995 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.280865+0000", 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3697999999999997e-05 2024-09-13T09:29:44.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:16.281027+0000", 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016203300000000001 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190202 3.9 3:95d5d7bd:::rbd_data.11189888a71f.0000000000000005:head [delete] snapc 8=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e204)", 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:39.702911+0000", 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8733716410000003, 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.50680710500000004, 2024-09-13T09:29:44.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190202 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.702911+0000", 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.702911+0000", 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.702907+0000", 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999962 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.702928+0000", 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0112999999999999e-05 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.702955+0000", 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7396e-05 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.702988+0000", 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2812999999999999e-05 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.703265+0000", 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027749300000000002 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:39.703310+0000", 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5176999999999998e-05 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.205621+0000", 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.502310378 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.205801+0000", 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018043099999999999 2024-09-13T09:29:44.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.205942+0000", 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014026899999999999 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.206987+0000", 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010458760000000001 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.209601+0000", 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0026140759999999999 2024-09-13T09:29:44.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.209670+0000", 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8376999999999999e-05 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.209718+0000", 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8433000000000002e-05 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.614 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190220 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:40.230792+0000", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3454906610000004, 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029259049999999999, 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190220 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230792+0000", 2024-09-13T09:29:44.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230792+0000", 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230790+0000", 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230803+0000", 2024-09-13T09:29:44.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3550000000000001e-05 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230835+0000", 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2021999999999999e-05 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230850+0000", 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.419e-05 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.230977+0000", 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000127207 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.231251+0000", 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027424799999999999 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.231748+0000", 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00049706499999999999 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.233333+0000", 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001584961 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.233467+0000", 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013437 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.233605+0000", 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013803700000000001 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:40.233718+0000", 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000112582 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190627 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.235169+0000", 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.341113623, 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006461484, 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190627 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235169+0000", 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235169+0000", 2024-09-13T09:29:44.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235167+0000", 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235180+0000", 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2754e-05 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235210+0000", 2024-09-13T09:29:44.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0015999999999999e-05 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235225+0000", 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4989e-05 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235315+0000", 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0014999999999998e-05 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235416+0000", 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010081600000000001 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.235596+0000", 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018019600000000001 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.236657+0000", 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010607839999999999 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.241371+0000", 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047142709999999999 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.241460+0000", 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9631999999999998e-05 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.241631+0000", 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017024499999999999 2024-09-13T09:29:44.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190648 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.265100+0000", 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.3111828489999999, 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028793320000000001, 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190648 2024-09-13T09:29:44.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265100+0000", 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265100+0000", 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265098+0000", 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265112+0000", 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4039000000000001e-05 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265147+0000", 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5366e-05 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265163+0000", 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5662999999999999e-05 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265293+0000", 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013024 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265450+0000", 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015736299999999999 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.265766+0000", 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031516699999999999 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.267168+0000", 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014021680000000001 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.267700+0000", 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00053194500000000003 2024-09-13T09:29:44.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.267790+0000", 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0457999999999993e-05 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.267979+0000", 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018911899999999999 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.631 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190655 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.277644+0000", 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2986389329999999, 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030129969999999999, 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190655 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277644+0000", 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277644+0000", 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277642+0000", 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:44.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277653+0000", 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0839000000000001e-05 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277677+0000", 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4477999999999998e-05 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277689+0000", 2024-09-13T09:29:44.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1776999999999999e-05 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277737+0000", 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7966999999999997e-05 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277795+0000", 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8258999999999999e-05 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.277915+0000", 2024-09-13T09:29:44.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012008899999999999 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.278871+0000", 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00095578899999999997 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.280520+0000", 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001648837 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.280570+0000", 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0318999999999997e-05 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.280657+0000", 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6625000000000005e-05 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190684 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.314308+0000", 2024-09-13T09:29:44.637 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.261974988, 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031185829999999999, 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190684 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314308+0000", 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314308+0000", 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314305+0000", 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314322+0000", 2024-09-13T09:29:44.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6849000000000001e-05 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314358+0000", 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6625999999999998e-05 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314375+0000", 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6549999999999999e-05 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314476+0000", 2024-09-13T09:29:44.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000100794 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.314639+0000", 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000162953 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.315006+0000", 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000367442 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.316546+0000", 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001539454 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.317231+0000", 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068559899999999995 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.317323+0000", 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1521999999999996e-05 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.317426+0000", 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103674 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190686 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.321291+0000", 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2549916379999999, 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030432139999999998, 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190686 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321291+0000", 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321291+0000", 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321289+0000", 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321302+0000", 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2897e-05 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321333+0000", 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1594999999999998e-05 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321351+0000", 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7224e-05 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321505+0000", 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015411799999999999 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.321730+0000", 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000225307 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.322215+0000", 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00048509900000000001 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.323427+0000", 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001211689 2024-09-13T09:29:44.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324157+0000", 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072977399999999998 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324215+0000", 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8004000000000002e-05 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324334+0000", 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000119764 2024-09-13T09:29:44.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190687 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.324507+0000", 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.251775847, 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040347730000000002, 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.649 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190687 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324507+0000", 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324507+0000", 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324505+0000", 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324514+0000", 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9500000000000007e-06 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324535+0000", 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1087e-05 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324551+0000", 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5426000000000001e-05 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324696+0000", 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014534699999999999 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.324967+0000", 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00027071400000000002 2024-09-13T09:29:44.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.325594+0000", 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00062747599999999997 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.327521+0000", 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019269230000000001 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.328377+0000", 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000856145 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.328468+0000", 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0383999999999994e-05 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.328542+0000", 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3868000000000005e-05 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190690 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.330500+0000", 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2457826439999999, 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029743780000000002, 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190690 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330500+0000", 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330500+0000", 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330497+0000", 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330512+0000", 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4260000000000001e-05 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330546+0000", 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4229e-05 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330573+0000", 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6937e-05 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330661+0000", 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7744000000000006e-05 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.330795+0000", 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013422400000000001 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.331139+0000", 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00034385900000000002 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.332455+0000", 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001316706 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.333390+0000", 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093455999999999997 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.333436+0000", 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5501999999999999e-05 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.333475+0000", 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9031e-05 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190706 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.660 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.356631+0000", 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2196515109999999, 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003155695, 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190706 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356631+0000", 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356631+0000", 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356630+0000", 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356639+0000", 2024-09-13T09:29:44.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3190000000000001e-06 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356659+0000", 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9831e-05 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356670+0000", 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0552e-05 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356773+0000", 2024-09-13T09:29:44.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103803 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.356967+0000", 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000194007 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.357368+0000", 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00040046599999999998 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.358748+0000", 2024-09-13T09:29:44.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013799089999999999 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.359487+0000", 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000739092 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.359588+0000", 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010066200000000001 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.359787+0000", 2024-09-13T09:29:44.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019938399999999999 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190715 3.1c 3:3fda0657:::rbd_object_map.111867511a81:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:43.373029+0000", 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.203253471, 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002947131, 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190715 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373029+0000", 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373029+0000", 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373028+0000", 2024-09-13T09:29:44.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373037+0000", 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1360000000000003e-06 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373053+0000", 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6578999999999998e-05 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373062+0000", 2024-09-13T09:29:44.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1130000000000001e-06 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373136+0000", 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373311+0000", 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00017573999999999999 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.373572+0000", 2024-09-13T09:29:44.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00026083399999999999 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.374833+0000", 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012608719999999999 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.375766+0000", 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00093319200000000001 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.375841+0000", 2024-09-13T09:29:44.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4305000000000001e-05 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:43.375976+0000", 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013584 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190823 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:44.313041+0000", 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.26324172099999998, 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030076669999999999, 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.671 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190823 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313041+0000", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313041+0000", 2024-09-13T09:29:44.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313039+0000", 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313052+0000", 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2887e-05 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313084+0000", 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2292000000000002e-05 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313099+0000", 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4562999999999999e-05 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313225+0000", 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012619399999999999 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313383+0000", 2024-09-13T09:29:44.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015842699999999999 2024-09-13T09:29:44.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:44.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.313738+0000", 2024-09-13T09:29:44.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035509499999999999 2024-09-13T09:29:44.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.315360+0000", 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016215229999999999 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.315976+0000", 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061633200000000003 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.316018+0000", 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2211e-05 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.316049+0000", 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0300000000000001e-05 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190825 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_remove_image in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:44.318395+0000", 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.25788776099999999, 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040001009999999998, 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190825 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318395+0000", 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318395+0000", 2024-09-13T09:29:44.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318393+0000", 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318406+0000", 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2442999999999999e-05 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318437+0000", 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1519000000000001e-05 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318451+0000", 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.411e-05 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318507+0000", 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5998000000000002e-05 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318560+0000", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3136000000000001e-05 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.318691+0000", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000130345 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.319711+0000", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010199829999999999 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.322156+0000", 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024449749999999998 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.322233+0000", 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7531e-05 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.322395+0000", 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000162034 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4376.0:190862 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_remove in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-13T09:29:44.377267+0000", 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.199015304, 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028812260000000002, 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4376", 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 190862 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-13T09:29:44.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377267+0000", 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377267+0000", 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377265+0000", 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-13T09:29:44.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377280+0000", 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4963999999999999e-05 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377310+0000", 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9989e-05 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377332+0000", 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1846000000000001e-05 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377437+0000", 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000105097 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377593+0000", 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015569100000000001 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.377889+0000", 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00029646300000000001 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.379281+0000", 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013917720000000001 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.380013+0000", 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00073189000000000001 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.380093+0000", 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0168999999999998e-05 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-13T09:29:44.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-13T09:29:44.380149+0000", 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5735000000000001e-05 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-13T09:29:44.689 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:44.689 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-13T09:29:44.689 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-13T09:29:44.689 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T09:29:44.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T09:29:44.691 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:29:44.692 INFO:tasks.ceph.osd.5:Started 2024-09-13T09:29:44.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:29:44.796 INFO:teuthology.orchestra.run.smithi099.stderr:no valid command found; 10 closest matches: 2024-09-13T09:29:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:0 2024-09-13T09:29:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:1 2024-09-13T09:29:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:2 2024-09-13T09:29:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:abort 2024-09-13T09:29:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:assert 2024-09-13T09:29:44.823 INFO:teuthology.orchestra.run.smithi099.stderr:config diff 2024-09-13T09:29:44.824 INFO:teuthology.orchestra.run.smithi099.stderr:config diff get 2024-09-13T09:29:44.824 INFO:teuthology.orchestra.run.smithi099.stderr:config get 2024-09-13T09:29:44.824 INFO:teuthology.orchestra.run.smithi099.stderr:config help [] 2024-09-13T09:29:44.824 INFO:teuthology.orchestra.run.smithi099.stderr:config set ... 2024-09-13T09:29:44.824 INFO:teuthology.orchestra.run.smithi099.stderr:admin_socket: invalid command 2024-09-13T09:29:44.825 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:29:44.826 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:44.826 INFO:teuthology.orchestra.run.smithi063.stdout: "num_ops": 0 2024-09-13T09:29:44.826 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:44.826 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-13T09:29:44.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:29:44.934 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [], 2024-09-13T09:29:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "complaint_time": 30, 2024-09-13T09:29:44.934 INFO:teuthology.orchestra.run.smithi063.stdout: "num_blocked_ops": 0 2024-09-13T09:29:44.934 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:44.944 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "size": 20, 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 600, 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "ops": [ 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:190202 3.9 e208/207 3:95d5d7bd:::rbd_data.11189888a71f.0000000000000005:head v 208'474, mlcod=0'0)", 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:40.205398+0000", 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 4.8438402429999998, 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0033296319999999999, 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.054 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205398+0000", 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205398+0000", 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205398+0000", 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.055 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205407+0000", 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.0920000000000004e-06 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205417+0000", 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0839000000000001e-05 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205431+0000", 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3232e-05 2024-09-13T09:29:45.056 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205512+0000", 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1321000000000003e-05 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.205552+0000", 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.9996999999999997e-05 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.057 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.208687+0000", 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.003134497 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:40.208728+0000", 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.1328999999999997e-05 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(client.4376.0:190624 3.0 e211/207 3:0220607e:::rbd_header.111867511a81:head v 211'1005, mlcod=210'1003)", 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.226627+0000", 2024-09-13T09:29:45.058 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.822611508, 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020535610000000002, 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.226627+0000", 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.226627+0000", 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.059 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.226626+0000", 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999995 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.226635+0000", 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1300000000000001e-06 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.226656+0000", 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1639999999999999e-05 2024-09-13T09:29:45.060 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.226670+0000", 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4093e-05 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.227539+0000", 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000868533 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.227558+0000", 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9228e-05 2024-09-13T09:29:45.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.228622+0000", 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010642500000000001 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.228680+0000", 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.8224999999999999e-05 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.062 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_repop(osd.0.0:3999 3.17 e211/208 3:ee110ec6:::rbd_data.111867511a81.0000000000000003:5 v 211'788, mlcod=189'785)", 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.275032+0000", 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.774206256, 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002111946, 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275032+0000", 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.063 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275032+0000", 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275031+0000", 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275043+0000", 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2091e-05 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.064 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275067+0000", 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.366e-05 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275084+0000", 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6860000000000001e-05 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275185+0000", 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000100832 2024-09-13T09:29:45.065 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.275231+0000", 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4.6480999999999998e-05 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.277069+0000", 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018377319999999999 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.277144+0000", 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.5092999999999994e-05 2024-09-13T09:29:45.066 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190739 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch watch cookie 140058279966640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.417891+0000", 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.63134724, 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022553339999999999, 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190739 2024-09-13T09:29:45.067 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.417891+0000", 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.417891+0000", 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.417889+0000", 2024-09-13T09:29:45.068 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.417902+0000", 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.3529999999999999e-05 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.417933+0000", 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0670000000000003e-05 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.069 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.417953+0000", 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9752999999999999e-05 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.418048+0000", 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5297999999999995e-05 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.418268+0000", 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00021973199999999999 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.070 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.419908+0000", 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016399330000000001 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.419986+0000", 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8719000000000003e-05 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.420147+0000", 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016012900000000001 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.071 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190744 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.427977+0000", 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.621261203, 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021332370000000001, 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190744 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.072 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.427977+0000", 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.427977+0000", 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.427975+0000", 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:29:45.073 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.427992+0000", 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7005999999999999e-05 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.428024+0000", 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2039999999999998e-05 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.428061+0000", 2024-09-13T09:29:45.074 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.7526e-05 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.428143+0000", 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1818999999999997e-05 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.075 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.428299+0000", 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015636799999999999 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.429910+0000", 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001610282 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.429994+0000", 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.4596000000000001e-05 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.076 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.430110+0000", 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000116233 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190754 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.439555+0000", 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6096830500000001, 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002194185, 2024-09-13T09:29:45.077 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190754 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439555+0000", 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.078 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439555+0000", 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439553+0000", 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439569+0000", 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5665999999999999e-05 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.079 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439599+0000", 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.0346999999999999e-05 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439615+0000", 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6133e-05 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439719+0000", 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000104045 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.080 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.439896+0000", 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00017662799999999999 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.441516+0000", 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0016198650000000001 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.441609+0000", 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.3252999999999996e-05 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.441750+0000", 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000140798 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190758 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.snapshot_trash_add in=21b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.444892+0000", 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6043466500000001, 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0027713109999999998, 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190758 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.083 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.444892+0000", 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.444892+0000", 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.444891+0000", 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:45.084 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.444897+0000", 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.5940000000000001e-06 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.444916+0000", 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9128999999999999e-05 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.085 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.444924+0000", 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.0129999999999993e-06 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445340+0000", 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00041649900000000003 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.086 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445509+0000", 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016831499999999999 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447390+0000", 2024-09-13T09:29:45.088 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0018817420000000001 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447498+0000", 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010796799999999999 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447663+0000", 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016480499999999999 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.089 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190759 3.18 3:1a426f9e:::rbd_header.11182e510213:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e211)", 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:43.445464+0000", 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 1.6037742310000001, 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024846880000000001, 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190759 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.090 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445464+0000", 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445464+0000", 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445462+0000", 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:45.091 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445477+0000", 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5316e-05 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445496+0000", 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8893999999999999e-05 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.445512+0000", 2024-09-13T09:29:45.092 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5625999999999999e-05 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.446920+0000", 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001408028 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for rw locks", 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447034+0000", 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000113935 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.093 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447704+0000", 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00067073600000000001 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447779+0000", 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4467000000000004e-05 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:43.447949+0000", 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000170082 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.094 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190765 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.224753+0000", 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.82448521299999999, 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020646800000000002, 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190765 2024-09-13T09:29:45.095 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224753+0000", 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224753+0000", 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224751+0000", 2024-09-13T09:29:45.096 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999976 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224766+0000", 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5138e-05 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224794+0000", 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.7840000000000001e-05 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.097 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224809+0000", 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4832e-05 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.224890+0000", 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.1380999999999999e-05 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.225016+0000", 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000126167 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.098 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.226564+0000", 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015480629999999999 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.226653+0000", 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8386000000000004e-05 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.226818+0000", 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00016533800000000001 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.099 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190769 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.235501+0000", 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.81373749200000001, 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0019938199999999999, 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.100 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190769 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235501+0000", 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235501+0000", 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.101 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235498+0000", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999971 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235513+0000", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4966e-05 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235545+0000", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.1266999999999997e-05 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.102 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235559+0000", 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4525999999999999e-05 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235674+0000", 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000114605 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.235829+0000", 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015505 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.103 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237278+0000", 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001449711 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237397+0000", 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000118272 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237495+0000", 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.8098999999999997e-05 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.104 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190771 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch unwatch cookie 140058279966640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.237795+0000", 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.81144338400000005, 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0024054860000000001, 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190771 2024-09-13T09:29:45.105 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237795+0000", 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237795+0000", 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237794+0000", 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.106 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237806+0000", 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.1948e-05 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237825+0000", 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.8926000000000001e-05 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.107 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.237840+0000", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.5639999999999999e-05 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238385+0000", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00054459599999999995 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238579+0000", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00019424899999999999 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.239854+0000", 2024-09-13T09:29:45.108 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012751710000000001 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.239975+0000", 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000120816 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.240201+0000", 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022545400000000001 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.109 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190772 3.18 3:1a426f9e:::rbd_header.11182e510213:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.238384+0000", 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.81085459500000001, 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002222178, 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190772 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.110 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238384+0000", 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238384+0000", 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238382+0000", 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.111 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238392+0000", 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.6439999999999995e-06 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238408+0000", 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6107999999999999e-05 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.238425+0000", 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6512e-05 2024-09-13T09:29:45.112 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.239435+0000", 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0010102729999999999 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for rw locks", 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.239609+0000", 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000174327 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.240245+0000", 2024-09-13T09:29:45.113 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00063544899999999995 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.240369+0000", 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00012460100000000001 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.240606+0000", 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023682 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.114 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190798 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch unwatch cookie 140058279664976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.281087+0000", 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.76815099499999995, 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022703939999999998, 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190798 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.115 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281087+0000", 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281087+0000", 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281087+0000", 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.116 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281093+0000", 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.2140000000000001e-06 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281102+0000", 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8680000000000005e-06 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281109+0000", 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.117 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281562+0000", 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000453523 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281734+0000", 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000172127 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.283071+0000", 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001336392 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.118 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.283136+0000", 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.5405000000000001e-05 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.283358+0000", 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00022180100000000001 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190799 3.18 3:1a426f9e:::rbd_header.11182e510213:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.281538+0000", 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.76770002199999998, 2024-09-13T09:29:45.119 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.002196656, 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190799 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281538+0000", 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.120 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281538+0000", 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281537+0000", 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281544+0000", 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.7939999999999997e-06 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.121 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281565+0000", 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1392000000000001e-05 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.281590+0000", 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.4664999999999998e-05 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.282776+0000", 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001185527 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.122 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for rw locks", 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.282906+0000", 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013078699999999999 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.283413+0000", 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00050667500000000005 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.283547+0000", 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013378899999999999 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.123 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.283735+0000", 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00018829900000000001 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190821 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch unwatch cookie 140058279966640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.309412+0000", 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.73982646200000002, 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022456920000000001, 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.124 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190821 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309412+0000", 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309412+0000", 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.125 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309411+0000", 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.999999 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309417+0000", 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 5.7799999999999997e-06 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309427+0000", 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.0715e-05 2024-09-13T09:29:45.126 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309437+0000", 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.7650000000000005e-06 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309689+0000", 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00025161399999999999 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309890+0000", 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000201485 2024-09-13T09:29:45.127 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.311232+0000", 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0013419339999999999 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.311338+0000", 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000105646 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.311658+0000", 2024-09-13T09:29:45.128 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00031972800000000003 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190822 3.18 3:1a426f9e:::rbd_header.11182e510213:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.309755+0000", 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.739483381, 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021789560000000001, 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.129 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190822 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309755+0000", 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309755+0000", 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.130 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309754+0000", 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309762+0000", 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.7399999999999993e-06 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309786+0000", 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.3266000000000001e-05 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.131 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.309794+0000", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.9139999999999997e-06 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.310910+0000", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001115046 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for rw locks", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.310978+0000", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 6.8683999999999995e-05 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.311709+0000", 2024-09-13T09:29:45.132 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00073038300000000003 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.311801+0000", 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.1895e-05 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.311934+0000", 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00013350800000000001 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.133 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190843 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.344739+0000", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.70449938599999995, 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020679639999999998, 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190843 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344739+0000", 2024-09-13T09:29:45.134 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344739+0000", 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344738+0000", 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344747+0000", 2024-09-13T09:29:45.135 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5670000000000001e-06 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344768+0000", 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 2.1121999999999999e-05 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344783+0000", 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.4620000000000001e-05 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.344887+0000", 2024-09-13T09:29:45.136 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010371599999999999 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.345050+0000", 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000162934 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.346606+0000", 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0015560109999999999 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.346701+0000", 2024-09-13T09:29:45.137 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.5013000000000007e-05 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.346807+0000", 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010645900000000001 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190853 3.18 3:1a426f9e:::rbd_header.11182e510213:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.357026+0000", 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.69221213199999998, 2024-09-13T09:29:45.138 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0020260090000000001, 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190853 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357026+0000", 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.139 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357026+0000", 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357024+0000", 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999981 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357037+0000", 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2608e-05 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.140 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357069+0000", 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 3.2116999999999998e-05 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357089+0000", 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.9751000000000001e-05 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357189+0000", 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 9.9706999999999999e-05 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.141 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.357344+0000", 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000155408 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.358876+0000", 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.001531768 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.358951+0000", 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.4939999999999997e-05 2024-09-13T09:29:45.142 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359052+0000", 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00010162099999999999 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190855 3.18 3:1a426f9e:::rbd_header.11182e510213:head [watch unwatch cookie 140058279664976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.359324+0000", 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.68991491599999999, 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0022314940000000001, 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.143 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190855 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359324+0000", 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.144 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359324+0000", 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359322+0000", 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359331+0000", 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 8.8619999999999992e-06 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.145 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359347+0000", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6005999999999999e-05 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359364+0000", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.6742e-05 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359857+0000", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00049324000000000004 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.360010+0000", 2024-09-13T09:29:45.146 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015291099999999999 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "op_commit", 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.361245+0000", 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0012350169999999999 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "commit_sent", 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.361324+0000", 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.8894999999999996e-05 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.361555+0000", 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00023105099999999999 2024-09-13T09:29:45.147 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "description": "osd_op(client.4376.0:190856 3.18 3:1a426f9e:::rbd_header.11182e510213:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e212)", 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "initiated_at": "2024-09-13T09:29:44.359898+0000", 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "age": 0.68934022100000003, 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.0021812300000000001, 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "type_data": { 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "flag_point": "started", 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_info": { 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client": "client.4376", 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "client_addr": "172.21.15.63:0/4127307939", 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: "tid": 190856 2024-09-13T09:29:45.148 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "events": [ 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "initiated", 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359898+0000", 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "throttled", 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359898+0000", 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "header_read", 2024-09-13T09:29:45.149 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359897+0000", 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 4294967295.9999986 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "all_read", 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359904+0000", 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 7.1999999999999997e-06 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "dispatched", 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359922+0000", 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.7527999999999999e-05 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.150 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "queued_for_pg", 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.359935+0000", 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 1.2918000000000001e-05 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.360731+0000", 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00079660799999999997 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "waiting for rw locks", 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.360881+0000", 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00015012200000000001 2024-09-13T09:29:45.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "reached_pg", 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.361621+0000", 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00073947599999999998 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "started", 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.361790+0000", 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.000169016 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "event": "done", 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "time": "2024-09-13T09:29:44.362079+0000", 2024-09-13T09:29:45.152 INFO:teuthology.orchestra.run.smithi063.stdout: "duration": 0.00028971200000000002 2024-09-13T09:29:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-09-13T09:29:45.153 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-09-13T09:29:45.153 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:29:45.244 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:29:45.315 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:29:45.312+0000 7fe6ed98c880 -1 Falling back to public interface 2024-09-13T09:29:46.250 INFO:teuthology.orchestra.run.smithi063.stderr:noscrub is unset 2024-09-13T09:29:46.265 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:29:47.256 INFO:teuthology.orchestra.run.smithi063.stderr:nodeep-scrub is unset 2024-09-13T09:29:49.153 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:29:49.152+0000 7fe6ed98c880 -1 osd.5 179 log_to_monitors true 2024-09-13T09:29:49.338 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-13T09:29:49.336+0000 7fe6e127a700 -1 osd.5 215 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:29:49.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:29:49.944 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-13T09:29:49.944 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-13T09:29:49.944 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-13T09:29:49.944 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-13T09:29:49.954 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T09:29:50.108 INFO:teuthology.orchestra.run.smithi063.stdout:mon.a: {} 2024-09-13T09:29:50.128 INFO:teuthology.orchestra.run.smithi063.stdout:mon.b: {} 2024-09-13T09:29:50.143 INFO:teuthology.orchestra.run.smithi063.stdout:mon.c: {} 2024-09-13T09:29:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:29:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:29:50.145 INFO:teuthology.orchestra.run.smithi063.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:29:50.158 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:29:51.278 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:29:51.295 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T09:29:52.278 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T09:29:52.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:29:53.315 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:29:53.327 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T09:29:54.321 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T09:29:54.335 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:29:55.331 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:29:55.344 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T09:29:56.346 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T09:29:56.360 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:29:57.353 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:29:57.366 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T09:29:58.357 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T09:29:58.371 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:29:59.372 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:29:59.387 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T09:30:00.203 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T09:30:00.215 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:30:01.390 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:30:01.405 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T09:30:02.398 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T09:30:02.415 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:30:03.411 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:30:03.426 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T09:30:04.426 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T09:30:04.442 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:30:05.441 INFO:teuthology.orchestra.run.smithi063.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:30:05.454 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T09:30:06.445 INFO:teuthology.orchestra.run.smithi063.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T09:30:06.457 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T09:30:06.457 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T09:30:06.457 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T09:30:06.457 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T09:30:06.457 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:30:06.458 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:30:06.764 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:06.764 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":232,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:30:06.428477+0000","last_up_change":"2024-09-13T09:29:50.256487+0000","last_in_change":"2024-09-13T09:29:08.613117+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T09:25:41.357433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T09:25:42.578616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:25:51.228779+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6817","nonce":2906107514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6819","nonce":2906107514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6823","nonce":2906107514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6821","nonce":2906107514}]},"public_addr":"172.21.15.63:6817/2906107514","cluster_addr":"172.21.15.63:6819/2906107514","heartbeat_back_addr":"172.21.15.63:6823/2906107514","heartbeat_front_addr":"172.21.15.63:6821/2906107514","state":["exists","up"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6825","nonce":3622854518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6827","nonce":3622854518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6831","nonce":3622854518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6829","nonce":3622854518}]},"public_addr":"172.21.15.63:6825/3622854518","cluster_addr":"172.21.15.63:6827/3622854518","heartbeat_back_addr":"172.21.15.63:6831/3622854518","heartbeat_front_addr":"172.21.15.63:6829/3622854518","state":["exists","up"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":194,"up_from":207,"up_thru":216,"down_at":196,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6801","nonce":2636060942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6803","nonce":2636060942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6807","nonce":2636060942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6805","nonce":2636060942}]},"public_addr":"172.21.15.63:6801/2636060942","cluster_addr":"172.21.15.63:6803/2636060942","heartbeat_back_addr":"172.21.15.63:6807/2636060942","heartbeat_front_addr":"172.21.15.63:6805/2636060942","state":["exists","up"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":46,"up_from":74,"up_thru":216,"down_at":48,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6809","nonce":4293004354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6811","nonce":4293004354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6815","nonce":4293004354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6813","nonce":4293004354}]},"public_addr":"172.21.15.63:6809/4293004354","cluster_addr":"172.21.15.63:6811/4293004354","heartbeat_back_addr":"172.21.15.63:6815/4293004354","heartbeat_front_addr":"172.21.15.63:6813/4293004354","state":["exists","up"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":142,"up_from":173,"up_thru":216,"down_at":144,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6801","nonce":4121301040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6803","nonce":4121301040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6807","nonce":4121301040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6805","nonce":4121301040}]},"public_addr":"172.21.15.99:6801/4121301040","cluster_addr":"172.21.15.99:6803/4121301040","heartbeat_back_addr":"172.21.15.99:6807/4121301040","heartbeat_front_addr":"172.21.15.99:6805/4121301040","state":["exists","up"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":179,"up_from":216,"up_thru":217,"down_at":181,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6809","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6811","nonce":1249382049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6813","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6815","nonce":1249382049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6823","nonce":1249382049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6819","nonce":1249382049}]},"public_addr":"172.21.15.99:6811/1249382049","cluster_addr":"172.21.15.99:6815/1249382049","heartbeat_back_addr":"172.21.15.99:6823/1249382049","heartbeat_front_addr":"172.21.15.99:6819/1249382049","state":["exists","up"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6810","nonce":4152410871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6814","nonce":4152410871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6822","nonce":4152410871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6818","nonce":4152410871}]},"public_addr":"172.21.15.99:6810/4152410871","cluster_addr":"172.21.15.99:6814/4152410871","heartbeat_back_addr":"172.21.15.99:6822/4152410871","heartbeat_front_addr":"172.21.15.99:6818/4152410871","state":["exists","up"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6825","nonce":2923799929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6827","nonce":2923799929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6831","nonce":2923799929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6829","nonce":2923799929}]},"public_addr":"172.21.15.99:6825/2923799929","cluster_addr":"172.21.15.99:6827/2923799929","heartbeat_back_addr":"172.21.15.99:6831/2923799929","heartbeat_front_addr":"172.21.15.99:6829/2923799929","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.267895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.161313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.447964+0000","dead_epoch":196},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.021320+0000","dead_epoch":48},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.883961+0000","dead_epoch":144},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.938146+0000","dead_epoch":181},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.627237+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.887523+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.18","osds":[2,5]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:30:06.780 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:30:06.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:30:07.089 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:07.089 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":232,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:30:06.428477+0000","last_up_change":"2024-09-13T09:29:50.256487+0000","last_in_change":"2024-09-13T09:29:08.613117+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T09:25:41.357433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T09:25:42.578616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:25:51.228779+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6817","nonce":2906107514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6819","nonce":2906107514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6823","nonce":2906107514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6821","nonce":2906107514}]},"public_addr":"172.21.15.63:6817/2906107514","cluster_addr":"172.21.15.63:6819/2906107514","heartbeat_back_addr":"172.21.15.63:6823/2906107514","heartbeat_front_addr":"172.21.15.63:6821/2906107514","state":["exists","up"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6825","nonce":3622854518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6827","nonce":3622854518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6831","nonce":3622854518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6829","nonce":3622854518}]},"public_addr":"172.21.15.63:6825/3622854518","cluster_addr":"172.21.15.63:6827/3622854518","heartbeat_back_addr":"172.21.15.63:6831/3622854518","heartbeat_front_addr":"172.21.15.63:6829/3622854518","state":["exists","up"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":194,"up_from":207,"up_thru":216,"down_at":196,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6801","nonce":2636060942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6803","nonce":2636060942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6807","nonce":2636060942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6805","nonce":2636060942}]},"public_addr":"172.21.15.63:6801/2636060942","cluster_addr":"172.21.15.63:6803/2636060942","heartbeat_back_addr":"172.21.15.63:6807/2636060942","heartbeat_front_addr":"172.21.15.63:6805/2636060942","state":["exists","up"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":46,"up_from":74,"up_thru":216,"down_at":48,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6809","nonce":4293004354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6811","nonce":4293004354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6815","nonce":4293004354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6813","nonce":4293004354}]},"public_addr":"172.21.15.63:6809/4293004354","cluster_addr":"172.21.15.63:6811/4293004354","heartbeat_back_addr":"172.21.15.63:6815/4293004354","heartbeat_front_addr":"172.21.15.63:6813/4293004354","state":["exists","up"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":142,"up_from":173,"up_thru":216,"down_at":144,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6801","nonce":4121301040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6803","nonce":4121301040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6807","nonce":4121301040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6805","nonce":4121301040}]},"public_addr":"172.21.15.99:6801/4121301040","cluster_addr":"172.21.15.99:6803/4121301040","heartbeat_back_addr":"172.21.15.99:6807/4121301040","heartbeat_front_addr":"172.21.15.99:6805/4121301040","state":["exists","up"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":179,"up_from":216,"up_thru":217,"down_at":181,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6809","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6811","nonce":1249382049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6813","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6815","nonce":1249382049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6823","nonce":1249382049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6819","nonce":1249382049}]},"public_addr":"172.21.15.99:6811/1249382049","cluster_addr":"172.21.15.99:6815/1249382049","heartbeat_back_addr":"172.21.15.99:6823/1249382049","heartbeat_front_addr":"172.21.15.99:6819/1249382049","state":["exists","up"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6810","nonce":4152410871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6814","nonce":4152410871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6822","nonce":4152410871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6818","nonce":4152410871}]},"public_addr":"172.21.15.99:6810/4152410871","cluster_addr":"172.21.15.99:6814/4152410871","heartbeat_back_addr":"172.21.15.99:6822/4152410871","heartbeat_front_addr":"172.21.15.99:6818/4152410871","state":["exists","up"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6825","nonce":2923799929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6827","nonce":2923799929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6831","nonce":2923799929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6829","nonce":2923799929}]},"public_addr":"172.21.15.99:6825/2923799929","cluster_addr":"172.21.15.99:6827/2923799929","heartbeat_back_addr":"172.21.15.99:6831/2923799929","heartbeat_front_addr":"172.21.15.99:6829/2923799929","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.267895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.161313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.447964+0000","dead_epoch":196},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.021320+0000","dead_epoch":48},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.883961+0000","dead_epoch":144},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.938146+0000","dead_epoch":181},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.627237+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.887523+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.18","osds":[2,5]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:30:07.104 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:30:07.104 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:30:07.104 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:30:07.105 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:30:07.105 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:30:07.106 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:30:07.106 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T09:30:07.107 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T09:30:07.321 INFO:teuthology.orchestra.run.smithi063.stdout:64424509496 2024-09-13T09:30:07.321 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:30:07.328 INFO:teuthology.orchestra.run.smithi063.stdout:317827579943 2024-09-13T09:30:07.328 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:30:07.329 INFO:teuthology.orchestra.run.smithi063.stdout:927712935941 2024-09-13T09:30:07.330 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:30:07.331 INFO:teuthology.orchestra.run.smithi063.stdout:64424509497 2024-09-13T09:30:07.331 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:30:07.339 INFO:teuthology.orchestra.run.smithi063.stdout:743029342225 2024-09-13T09:30:07.339 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:30:07.340 INFO:teuthology.orchestra.run.smithi063.stdout:64424509496 2024-09-13T09:30:07.340 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:30:07.345 INFO:teuthology.orchestra.run.smithi063.stdout:64424509497 2024-09-13T09:30:07.345 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:30:07.349 INFO:teuthology.orchestra.run.smithi063.stdout:889058230279 2024-09-13T09:30:07.349 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:30:07.816 INFO:teuthology.orchestra.run.smithi063.stdout:927712935940 2024-09-13T09:30:07.832 INFO:teuthology.orchestra.run.smithi063.stdout:64424509494 2024-09-13T09:30:07.836 INFO:teuthology.orchestra.run.smithi063.stdout:64424509495 2024-09-13T09:30:07.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 927712935941 got 927712935940 for osd.5 2024-09-13T09:30:07.844 INFO:teuthology.orchestra.run.smithi063.stdout:64424509494 2024-09-13T09:30:07.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509496 got 64424509494 for osd.0 2024-09-13T09:30:07.850 INFO:teuthology.orchestra.run.smithi063.stdout:64424509495 2024-09-13T09:30:07.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509495 for osd.1 2024-09-13T09:30:07.855 INFO:teuthology.orchestra.run.smithi063.stdout:743029342224 2024-09-13T09:30:07.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509496 got 64424509494 for osd.7 2024-09-13T09:30:07.866 INFO:teuthology.orchestra.run.smithi063.stdout:889058230278 2024-09-13T09:30:07.866 INFO:teuthology.orchestra.run.smithi063.stdout:317827579942 2024-09-13T09:30:07.869 INFO:tasks.ceph.ceph_manager.ceph:need seq 743029342225 got 743029342224 for osd.4 2024-09-13T09:30:07.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509495 for osd.6 2024-09-13T09:30:07.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 317827579943 got 317827579942 for osd.3 2024-09-13T09:30:07.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 889058230279 got 889058230278 for osd.2 2024-09-13T09:30:08.837 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:30:08.849 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:30:08.853 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:30:08.858 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:30:08.870 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:30:08.871 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:30:08.880 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:30:08.881 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:30:09.198 INFO:teuthology.orchestra.run.smithi063.stdout:927712935940 2024-09-13T09:30:09.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 927712935941 got 927712935940 for osd.5 2024-09-13T09:30:09.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-09-13T09:30:09.266+0000 7f49a087c700 0 --2- 172.21.15.63:0/221009185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f499c0ade40 0x7f499c1419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:30:09.338 INFO:teuthology.orchestra.run.smithi063.stdout:64424509495 2024-09-13T09:30:09.343 INFO:teuthology.orchestra.run.smithi063.stdout:64424509495 2024-09-13T09:30:09.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509495 for osd.6 2024-09-13T09:30:09.355 INFO:teuthology.orchestra.run.smithi063.stdout:64424509494 2024-09-13T09:30:09.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509495 for osd.1 2024-09-13T09:30:09.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509496 got 64424509494 for osd.0 2024-09-13T09:30:09.373 INFO:teuthology.orchestra.run.smithi063.stdout:317827579942 2024-09-13T09:30:09.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 317827579943 got 317827579942 for osd.3 2024-09-13T09:30:09.391 INFO:teuthology.orchestra.run.smithi063.stdout:743029342224 2024-09-13T09:30:09.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 743029342225 got 743029342224 for osd.4 2024-09-13T09:30:09.416 INFO:teuthology.orchestra.run.smithi063.stdout:64424509494 2024-09-13T09:30:09.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509496 got 64424509494 for osd.7 2024-09-13T09:30:09.520 INFO:teuthology.orchestra.run.smithi063.stdout:889058230279 2024-09-13T09:30:09.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 889058230279 got 889058230279 for osd.2 2024-09-13T09:30:09.535 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.212 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:30:10.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:30:10.360 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:30:10.373 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:30:10.389 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:30:10.404 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:30:10.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:30:10.667 INFO:teuthology.orchestra.run.smithi063.stdout:927712935941 2024-09-13T09:30:10.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 927712935941 got 927712935941 for osd.5 2024-09-13T09:30:10.686 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.755 INFO:teuthology.orchestra.run.smithi063.stdout:743029342226 2024-09-13T09:30:10.757 INFO:teuthology.orchestra.run.smithi063.stdout:64424509497 2024-09-13T09:30:10.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 743029342225 got 743029342226 for osd.4 2024-09-13T09:30:10.769 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509497 for osd.1 2024-09-13T09:30:10.770 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.798 INFO:teuthology.orchestra.run.smithi063.stdout:64424509497 2024-09-13T09:30:10.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509497 got 64424509497 for osd.6 2024-09-13T09:30:10.811 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.853 INFO:teuthology.orchestra.run.smithi063.stdout:64424509496 2024-09-13T09:30:10.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509496 got 64424509496 for osd.0 2024-09-13T09:30:10.868 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.871 INFO:teuthology.orchestra.run.smithi063.stdout:64424509496 2024-09-13T09:30:10.880 INFO:teuthology.orchestra.run.smithi063.stdout:317827579943 2024-09-13T09:30:10.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509496 got 64424509496 for osd.7 2024-09-13T09:30:10.884 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 317827579943 got 317827579943 for osd.3 2024-09-13T09:30:10.898 DEBUG:teuthology.parallel:result is None 2024-09-13T09:30:10.898 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:30:10.898 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:11.195 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:11.197 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:11.213 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-13T09:30:10.594659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25463,"ondisk_log_size":25463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395168,"kb_used_data":21980,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579552,"statfs":{"total":773094113280,"available":772689461248,"internally_reserved":0,"allocated":22507520,"data_stored":12029970,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.179818"},"pg_stats":[{"pgid":"3.1f","version":"217'535","reported_seq":2244,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700567+0000","last_change":"2024-09-13T09:29:51.329628+0000","last_active":"2024-09-13T09:30:04.700567+0000","last_peered":"2024-09-13T09:30:04.700567+0000","last_clean":"2024-09-13T09:30:04.700567+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:04.700567+0000","last_undegraded":"2024-09-13T09:30:04.700567+0000","last_fullsized":"2024-09-13T09:30:04.700567+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:43.571566+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"209'831","reported_seq":5831,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433026+0000","last_change":"2024-09-13T09:29:44.372628+0000","last_active":"2024-09-13T09:30:05.433026+0000","last_peered":"2024-09-13T09:30:05.433026+0000","last_clean":"2024-09-13T09:30:05.433026+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:05.433026+0000","last_undegraded":"2024-09-13T09:30:05.433026+0000","last_fullsized":"2024-09-13T09:30:05.433026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'629","reported_seq":5438,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748750+0000","last_change":"2024-09-13T09:29:52.302657+0000","last_active":"2024-09-13T09:30:06.748750+0000","last_peered":"2024-09-13T09:30:06.748750+0000","last_clean":"2024-09-13T09:30:06.748750+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:06.748750+0000","last_undegraded":"2024-09-13T09:30:06.748750+0000","last_fullsized":"2024-09-13T09:30:06.748750+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:50.114832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'819","reported_seq":6150,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748562+0000","last_change":"2024-09-13T09:29:52.330059+0000","last_active":"2024-09-13T09:30:06.748562+0000","last_peered":"2024-09-13T09:30:06.748562+0000","last_clean":"2024-09-13T09:30:06.748562+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:06.748562+0000","last_undegraded":"2024-09-13T09:30:06.748562+0000","last_fullsized":"2024-09-13T09:30:06.748562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:38.645385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7169,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700079+0000","last_change":"2024-09-13T09:29:44.278432+0000","last_active":"2024-09-13T09:30:04.700079+0000","last_peered":"2024-09-13T09:30:04.700079+0000","last_clean":"2024-09-13T09:30:04.700079+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:04.700079+0000","last_undegraded":"2024-09-13T09:30:04.700079+0000","last_fullsized":"2024-09-13T09:30:04.700079+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:24.917547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'680","reported_seq":13797,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434186+0000","last_change":"2024-09-13T09:29:44.237452+0000","last_active":"2024-09-13T09:30:05.434186+0000","last_peered":"2024-09-13T09:30:05.434186+0000","last_clean":"2024-09-13T09:30:05.434186+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:05.434186+0000","last_undegraded":"2024-09-13T09:30:05.434186+0000","last_fullsized":"2024-09-13T09:30:05.434186+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:22.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'666","reported_seq":3868,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246820+0000","last_change":"2024-09-13T09:29:44.322458+0000","last_active":"2024-09-13T09:30:06.246820+0000","last_peered":"2024-09-13T09:30:06.246820+0000","last_clean":"2024-09-13T09:30:06.246820+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:06.246820+0000","last_undegraded":"2024-09-13T09:30:06.246820+0000","last_fullsized":"2024-09-13T09:30:06.246820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"212'691","reported_seq":5866,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.746774+0000","last_change":"2024-09-13T09:29:51.341364+0000","last_active":"2024-09-13T09:30:06.746774+0000","last_peered":"2024-09-13T09:30:06.746774+0000","last_clean":"2024-09-13T09:30:06.746774+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:06.746774+0000","last_undegraded":"2024-09-13T09:30:06.746774+0000","last_fullsized":"2024-09-13T09:30:06.746774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:42.266961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"211'788","reported_seq":3015,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700432+0000","last_change":"2024-09-13T09:29:54.325150+0000","last_active":"2024-09-13T09:30:04.700432+0000","last_peered":"2024-09-13T09:30:04.700432+0000","last_clean":"2024-09-13T09:30:04.700432+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:04.700432+0000","last_undegraded":"2024-09-13T09:30:04.700432+0000","last_fullsized":"2024-09-13T09:30:04.700432+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:25.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6166,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748498+0000","last_change":"2024-09-13T09:29:51.321889+0000","last_active":"2024-09-13T09:30:06.748498+0000","last_peered":"2024-09-13T09:30:06.748498+0000","last_clean":"2024-09-13T09:30:06.748498+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:06.748498+0000","last_undegraded":"2024-09-13T09:30:06.748498+0000","last_fullsized":"2024-09-13T09:30:06.748498+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:12.873717+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'497","reported_seq":10587,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747787+0000","last_change":"2024-09-13T09:29:44.233280+0000","last_active":"2024-09-13T09:30:05.747787+0000","last_peered":"2024-09-13T09:30:05.747787+0000","last_clean":"2024-09-13T09:30:05.747787+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:05.747787+0000","last_undegraded":"2024-09-13T09:30:05.747787+0000","last_fullsized":"2024-09-13T09:30:05.747787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"210'613","reported_seq":6519,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434291+0000","last_change":"2024-09-13T09:29:44.238073+0000","last_active":"2024-09-13T09:30:05.434291+0000","last_peered":"2024-09-13T09:30:05.434291+0000","last_clean":"2024-09-13T09:30:05.434291+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:05.434291+0000","last_undegraded":"2024-09-13T09:30:05.434291+0000","last_fullsized":"2024-09-13T09:30:05.434291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:02.710053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":240,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748370+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:30:06.748370+0000","last_peered":"2024-09-13T09:30:06.748370+0000","last_clean":"2024-09-13T09:30:06.748370+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:06.748370+0000","last_undegraded":"2024-09-13T09:30:06.748370+0000","last_fullsized":"2024-09-13T09:30:06.748370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"217'540","reported_seq":2909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748496+0000","last_change":"2024-09-13T09:29:51.341689+0000","last_active":"2024-09-13T09:30:06.748496+0000","last_peered":"2024-09-13T09:30:06.748496+0000","last_clean":"2024-09-13T09:30:06.748496+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:06.748496+0000","last_undegraded":"2024-09-13T09:30:06.748496+0000","last_fullsized":"2024-09-13T09:30:06.748496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:56.843638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":248,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.749025+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:30:06.749025+0000","last_peered":"2024-09-13T09:30:06.749025+0000","last_clean":"2024-09-13T09:30:06.749025+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:06.749025+0000","last_undegraded":"2024-09-13T09:30:06.749025+0000","last_fullsized":"2024-09-13T09:30:06.749025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"210'965","reported_seq":4809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.610219+0000","last_change":"2024-09-13T09:29:44.322446+0000","last_active":"2024-09-13T09:30:06.610219+0000","last_peered":"2024-09-13T09:30:06.610219+0000","last_clean":"2024-09-13T09:30:06.610219+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:06.610219+0000","last_undegraded":"2024-09-13T09:30:06.610219+0000","last_fullsized":"2024-09-13T09:30:06.610219+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:22.045121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434427+0000","last_change":"2024-09-13T09:29:40.201770+0000","last_active":"2024-09-13T09:30:05.434427+0000","last_peered":"2024-09-13T09:30:05.434427+0000","last_clean":"2024-09-13T09:30:05.434427+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:05.434427+0000","last_undegraded":"2024-09-13T09:30:05.434427+0000","last_fullsized":"2024-09-13T09:30:05.434427+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:07.863545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9190,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433166+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:05.433166+0000","last_peered":"2024-09-13T09:30:05.433166+0000","last_clean":"2024-09-13T09:30:05.433166+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:05.433166+0000","last_undegraded":"2024-09-13T09:30:05.433166+0000","last_fullsized":"2024-09-13T09:30:05.433166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":200,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433208+0000","last_change":"2024-09-13T09:29:51.281204+0000","last_active":"2024-09-13T09:30:05.433208+0000","last_peered":"2024-09-13T09:30:05.433208+0000","last_clean":"2024-09-13T09:30:05.433208+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:05.433208+0000","last_undegraded":"2024-09-13T09:30:05.433208+0000","last_fullsized":"2024-09-13T09:30:05.433208+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:33.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'928","reported_seq":6275,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700427+0000","last_change":"2024-09-13T09:29:44.278448+0000","last_active":"2024-09-13T09:30:04.700427+0000","last_peered":"2024-09-13T09:30:04.700427+0000","last_clean":"2024-09-13T09:30:04.700427+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:04.700427+0000","last_undegraded":"2024-09-13T09:30:04.700427+0000","last_fullsized":"2024-09-13T09:30:04.700427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'1424","reported_seq":4477,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699603+0000","last_change":"2024-09-13T09:29:44.275636+0000","last_active":"2024-09-13T09:30:04.699603+0000","last_peered":"2024-09-13T09:30:04.699603+0000","last_clean":"2024-09-13T09:30:04.699603+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:04.699603+0000","last_undegraded":"2024-09-13T09:30:04.699603+0000","last_fullsized":"2024-09-13T09:30:04.699603+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:27.580162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":371,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747540+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:30:05.747540+0000","last_peered":"2024-09-13T09:30:05.747540+0000","last_clean":"2024-09-13T09:30:05.747540+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:05.747540+0000","last_undegraded":"2024-09-13T09:30:05.747540+0000","last_fullsized":"2024-09-13T09:30:05.747540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":229,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246866+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:30:06.246866+0000","last_peered":"2024-09-13T09:30:06.246866+0000","last_clean":"2024-09-13T09:30:06.246866+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:06.246866+0000","last_undegraded":"2024-09-13T09:30:06.246866+0000","last_fullsized":"2024-09-13T09:30:06.246866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9538,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246633+0000","last_change":"2024-09-13T09:29:44.321698+0000","last_active":"2024-09-13T09:30:06.246633+0000","last_peered":"2024-09-13T09:30:06.246633+0000","last_clean":"2024-09-13T09:30:06.246633+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:06.246633+0000","last_undegraded":"2024-09-13T09:30:06.246633+0000","last_fullsized":"2024-09-13T09:30:06.246633+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:11.466841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747653+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:30:05.747653+0000","last_peered":"2024-09-13T09:30:05.747653+0000","last_clean":"2024-09-13T09:30:05.747653+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:05.747653+0000","last_undegraded":"2024-09-13T09:30:05.747653+0000","last_fullsized":"2024-09-13T09:30:05.747653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"212'840","reported_seq":11367,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434421+0000","last_change":"2024-09-13T09:29:44.238109+0000","last_active":"2024-09-13T09:30:05.434421+0000","last_peered":"2024-09-13T09:30:05.434421+0000","last_clean":"2024-09-13T09:30:05.434421+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:05.434421+0000","last_undegraded":"2024-09-13T09:30:05.434421+0000","last_fullsized":"2024-09-13T09:30:05.434421+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.423203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":182,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748225+0000","last_change":"2024-09-13T09:29:51.279392+0000","last_active":"2024-09-13T09:30:06.748225+0000","last_peered":"2024-09-13T09:30:06.748225+0000","last_clean":"2024-09-13T09:30:06.748225+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:06.748225+0000","last_undegraded":"2024-09-13T09:30:06.748225+0000","last_fullsized":"2024-09-13T09:30:06.748225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:37.997623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7101,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432919+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:05.432919+0000","last_peered":"2024-09-13T09:30:05.432919+0000","last_clean":"2024-09-13T09:30:05.432919+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:05.432919+0000","last_undegraded":"2024-09-13T09:30:05.432919+0000","last_fullsized":"2024-09-13T09:30:05.432919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:28:54.042990+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747532+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:30:05.747532+0000","last_peered":"2024-09-13T09:30:05.747532+0000","last_clean":"2024-09-13T09:30:05.747532+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:05.747532+0000","last_undegraded":"2024-09-13T09:30:05.747532+0000","last_fullsized":"2024-09-13T09:30:05.747532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'653","reported_seq":3505,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432879+0000","last_change":"2024-09-13T09:29:44.370982+0000","last_active":"2024-09-13T09:30:05.432879+0000","last_peered":"2024-09-13T09:30:05.432879+0000","last_clean":"2024-09-13T09:30:05.432879+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:05.432879+0000","last_undegraded":"2024-09-13T09:30:05.432879+0000","last_fullsized":"2024-09-13T09:30:05.432879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"209'483","reported_seq":12049,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434041+0000","last_change":"2024-09-13T09:29:44.237484+0000","last_active":"2024-09-13T09:30:05.434041+0000","last_peered":"2024-09-13T09:30:05.434041+0000","last_clean":"2024-09-13T09:30:05.434041+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:05.434041+0000","last_undegraded":"2024-09-13T09:30:05.434041+0000","last_fullsized":"2024-09-13T09:30:05.434041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:37.361009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'572","reported_seq":7649,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748310+0000","last_change":"2024-09-13T09:29:44.229060+0000","last_active":"2024-09-13T09:30:06.748310+0000","last_peered":"2024-09-13T09:30:06.748310+0000","last_clean":"2024-09-13T09:30:06.748310+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:06.748310+0000","last_undegraded":"2024-09-13T09:30:06.748310+0000","last_fullsized":"2024-09-13T09:30:06.748310+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:07.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"210'622","reported_seq":4506,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433036+0000","last_change":"2024-09-13T09:29:44.371528+0000","last_active":"2024-09-13T09:30:05.433036+0000","last_peered":"2024-09-13T09:30:05.433036+0000","last_clean":"2024-09-13T09:30:05.433036+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:05.433036+0000","last_undegraded":"2024-09-13T09:30:05.433036+0000","last_fullsized":"2024-09-13T09:30:05.433036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7640,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748768+0000","last_change":"2024-09-13T09:29:51.357978+0000","last_active":"2024-09-13T09:30:06.748768+0000","last_peered":"2024-09-13T09:30:06.748768+0000","last_clean":"2024-09-13T09:30:06.748768+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:06.748768+0000","last_undegraded":"2024-09-13T09:30:06.748768+0000","last_fullsized":"2024-09-13T09:30:06.748768+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:32.572241+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"217'498","reported_seq":14666,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747792+0000","last_change":"2024-09-13T09:29:51.366598+0000","last_active":"2024-09-13T09:30:05.747792+0000","last_peered":"2024-09-13T09:30:05.747792+0000","last_clean":"2024-09-13T09:30:05.747792+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:05.747792+0000","last_undegraded":"2024-09-13T09:30:05.747792+0000","last_fullsized":"2024-09-13T09:30:05.747792+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:25:28.046650+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"211'1205","reported_seq":9159,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747948+0000","last_change":"2024-09-13T09:29:44.236105+0000","last_active":"2024-09-13T09:30:05.747948+0000","last_peered":"2024-09-13T09:30:05.747948+0000","last_clean":"2024-09-13T09:30:05.747948+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:05.747948+0000","last_undegraded":"2024-09-13T09:30:05.747948+0000","last_fullsized":"2024-09-13T09:30:05.747948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:30.493572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'483","reported_seq":7265,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747872+0000","last_change":"2024-09-13T09:29:44.236466+0000","last_active":"2024-09-13T09:30:05.747872+0000","last_peered":"2024-09-13T09:30:05.747872+0000","last_clean":"2024-09-13T09:30:05.747872+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:05.747872+0000","last_undegraded":"2024-09-13T09:30:05.747872+0000","last_fullsized":"2024-09-13T09:30:05.747872+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:24.511124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'887","reported_seq":4070,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748769+0000","last_change":"2024-09-13T09:29:44.229062+0000","last_active":"2024-09-13T09:30:06.748769+0000","last_peered":"2024-09-13T09:30:06.748769+0000","last_clean":"2024-09-13T09:30:06.748769+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:06.748769+0000","last_undegraded":"2024-09-13T09:30:06.748769+0000","last_fullsized":"2024-09-13T09:30:06.748769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"212'895","reported_seq":9205,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747542+0000","last_change":"2024-09-13T09:29:44.235484+0000","last_active":"2024-09-13T09:30:05.747542+0000","last_peered":"2024-09-13T09:30:05.747542+0000","last_clean":"2024-09-13T09:30:05.747542+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:05.747542+0000","last_undegraded":"2024-09-13T09:30:05.747542+0000","last_fullsized":"2024-09-13T09:30:05.747542+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:42.588638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6241,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699859+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:04.699859+0000","last_peered":"2024-09-13T09:30:04.699859+0000","last_clean":"2024-09-13T09:30:04.699859+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:04.699859+0000","last_undegraded":"2024-09-13T09:30:04.699859+0000","last_fullsized":"2024-09-13T09:30:04.699859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"209'1122","reported_seq":9794,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747717+0000","last_change":"2024-09-13T09:29:44.234845+0000","last_active":"2024-09-13T09:30:05.747717+0000","last_peered":"2024-09-13T09:30:05.747717+0000","last_clean":"2024-09-13T09:30:05.747717+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:05.747717+0000","last_undegraded":"2024-09-13T09:30:05.747717+0000","last_fullsized":"2024-09-13T09:30:05.747717+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:30.144378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25343,"ondisk_log_size":25343,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51976,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.099},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.145}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.1799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509497,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55976,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315864,"statfs":{"total":96636764160,"available":96579444736,"internally_reserved":0,"allocated":2777088,"data_stored":1346707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":1.621}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":1.165},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.393}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":6.9450000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.68799999999999994}]}]},{"osd":5,"up_from":216,"seq":927712935942,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":173,"seq":743029342226,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.712},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.7989999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.534},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1060000000000001}]}]},{"osd":3,"up_from":74,"seq":317827579943,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79924,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291916,"statfs":{"total":96636764160,"available":96554921984,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0529999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.749}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.02},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.099},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.355},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.391}]}]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.246}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.155},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":1.5549999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.286}]}]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.876}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.117},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":0.752},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":1.2250000000000001}]}]},{"osd":2,"up_from":207,"seq":889058230280,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:11.216 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:11.517 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:11.520 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:11.533 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-13T09:30:10.594659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25463,"ondisk_log_size":25463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395168,"kb_used_data":21980,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579552,"statfs":{"total":773094113280,"available":772689461248,"internally_reserved":0,"allocated":22507520,"data_stored":12029970,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.179818"},"pg_stats":[{"pgid":"3.1f","version":"217'535","reported_seq":2244,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700567+0000","last_change":"2024-09-13T09:29:51.329628+0000","last_active":"2024-09-13T09:30:04.700567+0000","last_peered":"2024-09-13T09:30:04.700567+0000","last_clean":"2024-09-13T09:30:04.700567+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:04.700567+0000","last_undegraded":"2024-09-13T09:30:04.700567+0000","last_fullsized":"2024-09-13T09:30:04.700567+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:43.571566+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"209'831","reported_seq":5831,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433026+0000","last_change":"2024-09-13T09:29:44.372628+0000","last_active":"2024-09-13T09:30:05.433026+0000","last_peered":"2024-09-13T09:30:05.433026+0000","last_clean":"2024-09-13T09:30:05.433026+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:05.433026+0000","last_undegraded":"2024-09-13T09:30:05.433026+0000","last_fullsized":"2024-09-13T09:30:05.433026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'629","reported_seq":5438,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748750+0000","last_change":"2024-09-13T09:29:52.302657+0000","last_active":"2024-09-13T09:30:06.748750+0000","last_peered":"2024-09-13T09:30:06.748750+0000","last_clean":"2024-09-13T09:30:06.748750+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:06.748750+0000","last_undegraded":"2024-09-13T09:30:06.748750+0000","last_fullsized":"2024-09-13T09:30:06.748750+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:50.114832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'819","reported_seq":6150,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748562+0000","last_change":"2024-09-13T09:29:52.330059+0000","last_active":"2024-09-13T09:30:06.748562+0000","last_peered":"2024-09-13T09:30:06.748562+0000","last_clean":"2024-09-13T09:30:06.748562+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:06.748562+0000","last_undegraded":"2024-09-13T09:30:06.748562+0000","last_fullsized":"2024-09-13T09:30:06.748562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:38.645385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7169,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700079+0000","last_change":"2024-09-13T09:29:44.278432+0000","last_active":"2024-09-13T09:30:04.700079+0000","last_peered":"2024-09-13T09:30:04.700079+0000","last_clean":"2024-09-13T09:30:04.700079+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:04.700079+0000","last_undegraded":"2024-09-13T09:30:04.700079+0000","last_fullsized":"2024-09-13T09:30:04.700079+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:24.917547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'680","reported_seq":13797,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434186+0000","last_change":"2024-09-13T09:29:44.237452+0000","last_active":"2024-09-13T09:30:05.434186+0000","last_peered":"2024-09-13T09:30:05.434186+0000","last_clean":"2024-09-13T09:30:05.434186+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:05.434186+0000","last_undegraded":"2024-09-13T09:30:05.434186+0000","last_fullsized":"2024-09-13T09:30:05.434186+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:22.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'666","reported_seq":3868,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246820+0000","last_change":"2024-09-13T09:29:44.322458+0000","last_active":"2024-09-13T09:30:06.246820+0000","last_peered":"2024-09-13T09:30:06.246820+0000","last_clean":"2024-09-13T09:30:06.246820+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:06.246820+0000","last_undegraded":"2024-09-13T09:30:06.246820+0000","last_fullsized":"2024-09-13T09:30:06.246820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"212'691","reported_seq":5866,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.746774+0000","last_change":"2024-09-13T09:29:51.341364+0000","last_active":"2024-09-13T09:30:06.746774+0000","last_peered":"2024-09-13T09:30:06.746774+0000","last_clean":"2024-09-13T09:30:06.746774+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:06.746774+0000","last_undegraded":"2024-09-13T09:30:06.746774+0000","last_fullsized":"2024-09-13T09:30:06.746774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:42.266961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"211'788","reported_seq":3015,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700432+0000","last_change":"2024-09-13T09:29:54.325150+0000","last_active":"2024-09-13T09:30:04.700432+0000","last_peered":"2024-09-13T09:30:04.700432+0000","last_clean":"2024-09-13T09:30:04.700432+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:04.700432+0000","last_undegraded":"2024-09-13T09:30:04.700432+0000","last_fullsized":"2024-09-13T09:30:04.700432+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:25.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6166,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748498+0000","last_change":"2024-09-13T09:29:51.321889+0000","last_active":"2024-09-13T09:30:06.748498+0000","last_peered":"2024-09-13T09:30:06.748498+0000","last_clean":"2024-09-13T09:30:06.748498+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:06.748498+0000","last_undegraded":"2024-09-13T09:30:06.748498+0000","last_fullsized":"2024-09-13T09:30:06.748498+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:12.873717+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'497","reported_seq":10587,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747787+0000","last_change":"2024-09-13T09:29:44.233280+0000","last_active":"2024-09-13T09:30:05.747787+0000","last_peered":"2024-09-13T09:30:05.747787+0000","last_clean":"2024-09-13T09:30:05.747787+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:05.747787+0000","last_undegraded":"2024-09-13T09:30:05.747787+0000","last_fullsized":"2024-09-13T09:30:05.747787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"210'613","reported_seq":6519,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434291+0000","last_change":"2024-09-13T09:29:44.238073+0000","last_active":"2024-09-13T09:30:05.434291+0000","last_peered":"2024-09-13T09:30:05.434291+0000","last_clean":"2024-09-13T09:30:05.434291+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:05.434291+0000","last_undegraded":"2024-09-13T09:30:05.434291+0000","last_fullsized":"2024-09-13T09:30:05.434291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:02.710053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":240,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748370+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:30:06.748370+0000","last_peered":"2024-09-13T09:30:06.748370+0000","last_clean":"2024-09-13T09:30:06.748370+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:06.748370+0000","last_undegraded":"2024-09-13T09:30:06.748370+0000","last_fullsized":"2024-09-13T09:30:06.748370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"217'540","reported_seq":2909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748496+0000","last_change":"2024-09-13T09:29:51.341689+0000","last_active":"2024-09-13T09:30:06.748496+0000","last_peered":"2024-09-13T09:30:06.748496+0000","last_clean":"2024-09-13T09:30:06.748496+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:06.748496+0000","last_undegraded":"2024-09-13T09:30:06.748496+0000","last_fullsized":"2024-09-13T09:30:06.748496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:56.843638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":248,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.749025+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:30:06.749025+0000","last_peered":"2024-09-13T09:30:06.749025+0000","last_clean":"2024-09-13T09:30:06.749025+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:06.749025+0000","last_undegraded":"2024-09-13T09:30:06.749025+0000","last_fullsized":"2024-09-13T09:30:06.749025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"210'965","reported_seq":4809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.610219+0000","last_change":"2024-09-13T09:29:44.322446+0000","last_active":"2024-09-13T09:30:06.610219+0000","last_peered":"2024-09-13T09:30:06.610219+0000","last_clean":"2024-09-13T09:30:06.610219+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:06.610219+0000","last_undegraded":"2024-09-13T09:30:06.610219+0000","last_fullsized":"2024-09-13T09:30:06.610219+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:22.045121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434427+0000","last_change":"2024-09-13T09:29:40.201770+0000","last_active":"2024-09-13T09:30:05.434427+0000","last_peered":"2024-09-13T09:30:05.434427+0000","last_clean":"2024-09-13T09:30:05.434427+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:05.434427+0000","last_undegraded":"2024-09-13T09:30:05.434427+0000","last_fullsized":"2024-09-13T09:30:05.434427+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:07.863545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9190,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433166+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:05.433166+0000","last_peered":"2024-09-13T09:30:05.433166+0000","last_clean":"2024-09-13T09:30:05.433166+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:05.433166+0000","last_undegraded":"2024-09-13T09:30:05.433166+0000","last_fullsized":"2024-09-13T09:30:05.433166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":200,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433208+0000","last_change":"2024-09-13T09:29:51.281204+0000","last_active":"2024-09-13T09:30:05.433208+0000","last_peered":"2024-09-13T09:30:05.433208+0000","last_clean":"2024-09-13T09:30:05.433208+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:05.433208+0000","last_undegraded":"2024-09-13T09:30:05.433208+0000","last_fullsized":"2024-09-13T09:30:05.433208+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:33.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'928","reported_seq":6275,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700427+0000","last_change":"2024-09-13T09:29:44.278448+0000","last_active":"2024-09-13T09:30:04.700427+0000","last_peered":"2024-09-13T09:30:04.700427+0000","last_clean":"2024-09-13T09:30:04.700427+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:04.700427+0000","last_undegraded":"2024-09-13T09:30:04.700427+0000","last_fullsized":"2024-09-13T09:30:04.700427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'1424","reported_seq":4477,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699603+0000","last_change":"2024-09-13T09:29:44.275636+0000","last_active":"2024-09-13T09:30:04.699603+0000","last_peered":"2024-09-13T09:30:04.699603+0000","last_clean":"2024-09-13T09:30:04.699603+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:04.699603+0000","last_undegraded":"2024-09-13T09:30:04.699603+0000","last_fullsized":"2024-09-13T09:30:04.699603+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:27.580162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":371,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747540+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:30:05.747540+0000","last_peered":"2024-09-13T09:30:05.747540+0000","last_clean":"2024-09-13T09:30:05.747540+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:05.747540+0000","last_undegraded":"2024-09-13T09:30:05.747540+0000","last_fullsized":"2024-09-13T09:30:05.747540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":229,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246866+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:30:06.246866+0000","last_peered":"2024-09-13T09:30:06.246866+0000","last_clean":"2024-09-13T09:30:06.246866+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:06.246866+0000","last_undegraded":"2024-09-13T09:30:06.246866+0000","last_fullsized":"2024-09-13T09:30:06.246866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9538,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246633+0000","last_change":"2024-09-13T09:29:44.321698+0000","last_active":"2024-09-13T09:30:06.246633+0000","last_peered":"2024-09-13T09:30:06.246633+0000","last_clean":"2024-09-13T09:30:06.246633+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:06.246633+0000","last_undegraded":"2024-09-13T09:30:06.246633+0000","last_fullsized":"2024-09-13T09:30:06.246633+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:11.466841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747653+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:30:05.747653+0000","last_peered":"2024-09-13T09:30:05.747653+0000","last_clean":"2024-09-13T09:30:05.747653+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:05.747653+0000","last_undegraded":"2024-09-13T09:30:05.747653+0000","last_fullsized":"2024-09-13T09:30:05.747653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"212'840","reported_seq":11367,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434421+0000","last_change":"2024-09-13T09:29:44.238109+0000","last_active":"2024-09-13T09:30:05.434421+0000","last_peered":"2024-09-13T09:30:05.434421+0000","last_clean":"2024-09-13T09:30:05.434421+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:05.434421+0000","last_undegraded":"2024-09-13T09:30:05.434421+0000","last_fullsized":"2024-09-13T09:30:05.434421+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.423203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":182,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748225+0000","last_change":"2024-09-13T09:29:51.279392+0000","last_active":"2024-09-13T09:30:06.748225+0000","last_peered":"2024-09-13T09:30:06.748225+0000","last_clean":"2024-09-13T09:30:06.748225+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:06.748225+0000","last_undegraded":"2024-09-13T09:30:06.748225+0000","last_fullsized":"2024-09-13T09:30:06.748225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:37.997623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7101,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432919+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:05.432919+0000","last_peered":"2024-09-13T09:30:05.432919+0000","last_clean":"2024-09-13T09:30:05.432919+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:05.432919+0000","last_undegraded":"2024-09-13T09:30:05.432919+0000","last_fullsized":"2024-09-13T09:30:05.432919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:28:54.042990+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747532+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:30:05.747532+0000","last_peered":"2024-09-13T09:30:05.747532+0000","last_clean":"2024-09-13T09:30:05.747532+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:05.747532+0000","last_undegraded":"2024-09-13T09:30:05.747532+0000","last_fullsized":"2024-09-13T09:30:05.747532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'653","reported_seq":3505,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432879+0000","last_change":"2024-09-13T09:29:44.370982+0000","last_active":"2024-09-13T09:30:05.432879+0000","last_peered":"2024-09-13T09:30:05.432879+0000","last_clean":"2024-09-13T09:30:05.432879+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:05.432879+0000","last_undegraded":"2024-09-13T09:30:05.432879+0000","last_fullsized":"2024-09-13T09:30:05.432879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"209'483","reported_seq":12049,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434041+0000","last_change":"2024-09-13T09:29:44.237484+0000","last_active":"2024-09-13T09:30:05.434041+0000","last_peered":"2024-09-13T09:30:05.434041+0000","last_clean":"2024-09-13T09:30:05.434041+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:05.434041+0000","last_undegraded":"2024-09-13T09:30:05.434041+0000","last_fullsized":"2024-09-13T09:30:05.434041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:37.361009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'572","reported_seq":7649,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748310+0000","last_change":"2024-09-13T09:29:44.229060+0000","last_active":"2024-09-13T09:30:06.748310+0000","last_peered":"2024-09-13T09:30:06.748310+0000","last_clean":"2024-09-13T09:30:06.748310+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:06.748310+0000","last_undegraded":"2024-09-13T09:30:06.748310+0000","last_fullsized":"2024-09-13T09:30:06.748310+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:07.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"210'622","reported_seq":4506,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433036+0000","last_change":"2024-09-13T09:29:44.371528+0000","last_active":"2024-09-13T09:30:05.433036+0000","last_peered":"2024-09-13T09:30:05.433036+0000","last_clean":"2024-09-13T09:30:05.433036+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:05.433036+0000","last_undegraded":"2024-09-13T09:30:05.433036+0000","last_fullsized":"2024-09-13T09:30:05.433036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7640,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748768+0000","last_change":"2024-09-13T09:29:51.357978+0000","last_active":"2024-09-13T09:30:06.748768+0000","last_peered":"2024-09-13T09:30:06.748768+0000","last_clean":"2024-09-13T09:30:06.748768+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:06.748768+0000","last_undegraded":"2024-09-13T09:30:06.748768+0000","last_fullsized":"2024-09-13T09:30:06.748768+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:32.572241+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"217'498","reported_seq":14666,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747792+0000","last_change":"2024-09-13T09:29:51.366598+0000","last_active":"2024-09-13T09:30:05.747792+0000","last_peered":"2024-09-13T09:30:05.747792+0000","last_clean":"2024-09-13T09:30:05.747792+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:05.747792+0000","last_undegraded":"2024-09-13T09:30:05.747792+0000","last_fullsized":"2024-09-13T09:30:05.747792+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:25:28.046650+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"211'1205","reported_seq":9159,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747948+0000","last_change":"2024-09-13T09:29:44.236105+0000","last_active":"2024-09-13T09:30:05.747948+0000","last_peered":"2024-09-13T09:30:05.747948+0000","last_clean":"2024-09-13T09:30:05.747948+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:05.747948+0000","last_undegraded":"2024-09-13T09:30:05.747948+0000","last_fullsized":"2024-09-13T09:30:05.747948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:30.493572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'483","reported_seq":7265,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747872+0000","last_change":"2024-09-13T09:29:44.236466+0000","last_active":"2024-09-13T09:30:05.747872+0000","last_peered":"2024-09-13T09:30:05.747872+0000","last_clean":"2024-09-13T09:30:05.747872+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:05.747872+0000","last_undegraded":"2024-09-13T09:30:05.747872+0000","last_fullsized":"2024-09-13T09:30:05.747872+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:24.511124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'887","reported_seq":4070,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748769+0000","last_change":"2024-09-13T09:29:44.229062+0000","last_active":"2024-09-13T09:30:06.748769+0000","last_peered":"2024-09-13T09:30:06.748769+0000","last_clean":"2024-09-13T09:30:06.748769+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:06.748769+0000","last_undegraded":"2024-09-13T09:30:06.748769+0000","last_fullsized":"2024-09-13T09:30:06.748769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"212'895","reported_seq":9205,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747542+0000","last_change":"2024-09-13T09:29:44.235484+0000","last_active":"2024-09-13T09:30:05.747542+0000","last_peered":"2024-09-13T09:30:05.747542+0000","last_clean":"2024-09-13T09:30:05.747542+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:05.747542+0000","last_undegraded":"2024-09-13T09:30:05.747542+0000","last_fullsized":"2024-09-13T09:30:05.747542+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:42.588638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6241,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699859+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:04.699859+0000","last_peered":"2024-09-13T09:30:04.699859+0000","last_clean":"2024-09-13T09:30:04.699859+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:04.699859+0000","last_undegraded":"2024-09-13T09:30:04.699859+0000","last_fullsized":"2024-09-13T09:30:04.699859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"209'1122","reported_seq":9794,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747717+0000","last_change":"2024-09-13T09:29:44.234845+0000","last_active":"2024-09-13T09:30:05.747717+0000","last_peered":"2024-09-13T09:30:05.747717+0000","last_clean":"2024-09-13T09:30:05.747717+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:05.747717+0000","last_undegraded":"2024-09-13T09:30:05.747717+0000","last_fullsized":"2024-09-13T09:30:05.747717+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:30.144378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25343,"ondisk_log_size":25343,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51976,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.099},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.145}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.1799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509497,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55976,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315864,"statfs":{"total":96636764160,"available":96579444736,"internally_reserved":0,"allocated":2777088,"data_stored":1346707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":1.621}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":1.165},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.393}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":6.9450000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.68799999999999994}]}]},{"osd":5,"up_from":216,"seq":927712935942,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":173,"seq":743029342226,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.712},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.7989999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.534},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1060000000000001}]}]},{"osd":3,"up_from":74,"seq":317827579943,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79924,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291916,"statfs":{"total":96636764160,"available":96554921984,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0529999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.749}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.02},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.099},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.355},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.391}]}]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.246}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.155},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":1.5549999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.286}]}]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.876}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.117},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":0.752},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":1.2250000000000001}]}]},{"osd":2,"up_from":207,"seq":889058230280,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:11.536 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:30:11.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T09:30:11.854 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T09:30:11.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:30:11.863 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:12.174 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:12.176 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:12.189 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-13T09:30:10.594659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25463,"ondisk_log_size":25463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395168,"kb_used_data":21980,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579552,"statfs":{"total":773094113280,"available":772689461248,"internally_reserved":0,"allocated":22507520,"data_stored":12029970,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.179818"},"pg_stats":[{"pgid":"3.1f","version":"217'535","reported_seq":2244,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700567+0000","last_change":"2024-09-13T09:29:51.329628+0000","last_active":"2024-09-13T09:30:04.700567+0000","last_peered":"2024-09-13T09:30:04.700567+0000","last_clean":"2024-09-13T09:30:04.700567+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:04.700567+0000","last_undegraded":"2024-09-13T09:30:04.700567+0000","last_fullsized":"2024-09-13T09:30:04.700567+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:43.571566+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"209'831","reported_seq":5831,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433026+0000","last_change":"2024-09-13T09:29:44.372628+0000","last_active":"2024-09-13T09:30:05.433026+0000","last_peered":"2024-09-13T09:30:05.433026+0000","last_clean":"2024-09-13T09:30:05.433026+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:05.433026+0000","last_undegraded":"2024-09-13T09:30:05.433026+0000","last_fullsized":"2024-09-13T09:30:05.433026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'629","reported_seq":5438,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748750+0000","last_change":"2024-09-13T09:29:52.302657+0000","last_active":"2024-09-13T09:30:06.748750+0000","last_peered":"2024-09-13T09:30:06.748750+0000","last_clean":"2024-09-13T09:30:06.748750+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:06.748750+0000","last_undegraded":"2024-09-13T09:30:06.748750+0000","last_fullsized":"2024-09-13T09:30:06.748750+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:50.114832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'819","reported_seq":6150,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748562+0000","last_change":"2024-09-13T09:29:52.330059+0000","last_active":"2024-09-13T09:30:06.748562+0000","last_peered":"2024-09-13T09:30:06.748562+0000","last_clean":"2024-09-13T09:30:06.748562+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:06.748562+0000","last_undegraded":"2024-09-13T09:30:06.748562+0000","last_fullsized":"2024-09-13T09:30:06.748562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:38.645385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7169,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700079+0000","last_change":"2024-09-13T09:29:44.278432+0000","last_active":"2024-09-13T09:30:04.700079+0000","last_peered":"2024-09-13T09:30:04.700079+0000","last_clean":"2024-09-13T09:30:04.700079+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:04.700079+0000","last_undegraded":"2024-09-13T09:30:04.700079+0000","last_fullsized":"2024-09-13T09:30:04.700079+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:24.917547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'680","reported_seq":13797,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434186+0000","last_change":"2024-09-13T09:29:44.237452+0000","last_active":"2024-09-13T09:30:05.434186+0000","last_peered":"2024-09-13T09:30:05.434186+0000","last_clean":"2024-09-13T09:30:05.434186+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:05.434186+0000","last_undegraded":"2024-09-13T09:30:05.434186+0000","last_fullsized":"2024-09-13T09:30:05.434186+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:22.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'666","reported_seq":3868,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246820+0000","last_change":"2024-09-13T09:29:44.322458+0000","last_active":"2024-09-13T09:30:06.246820+0000","last_peered":"2024-09-13T09:30:06.246820+0000","last_clean":"2024-09-13T09:30:06.246820+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:06.246820+0000","last_undegraded":"2024-09-13T09:30:06.246820+0000","last_fullsized":"2024-09-13T09:30:06.246820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"212'691","reported_seq":5866,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.746774+0000","last_change":"2024-09-13T09:29:51.341364+0000","last_active":"2024-09-13T09:30:06.746774+0000","last_peered":"2024-09-13T09:30:06.746774+0000","last_clean":"2024-09-13T09:30:06.746774+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:06.746774+0000","last_undegraded":"2024-09-13T09:30:06.746774+0000","last_fullsized":"2024-09-13T09:30:06.746774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:42.266961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"211'788","reported_seq":3015,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700432+0000","last_change":"2024-09-13T09:29:54.325150+0000","last_active":"2024-09-13T09:30:04.700432+0000","last_peered":"2024-09-13T09:30:04.700432+0000","last_clean":"2024-09-13T09:30:04.700432+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:04.700432+0000","last_undegraded":"2024-09-13T09:30:04.700432+0000","last_fullsized":"2024-09-13T09:30:04.700432+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:25.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6166,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748498+0000","last_change":"2024-09-13T09:29:51.321889+0000","last_active":"2024-09-13T09:30:06.748498+0000","last_peered":"2024-09-13T09:30:06.748498+0000","last_clean":"2024-09-13T09:30:06.748498+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:06.748498+0000","last_undegraded":"2024-09-13T09:30:06.748498+0000","last_fullsized":"2024-09-13T09:30:06.748498+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:12.873717+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'497","reported_seq":10587,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747787+0000","last_change":"2024-09-13T09:29:44.233280+0000","last_active":"2024-09-13T09:30:05.747787+0000","last_peered":"2024-09-13T09:30:05.747787+0000","last_clean":"2024-09-13T09:30:05.747787+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:05.747787+0000","last_undegraded":"2024-09-13T09:30:05.747787+0000","last_fullsized":"2024-09-13T09:30:05.747787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"210'613","reported_seq":6519,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434291+0000","last_change":"2024-09-13T09:29:44.238073+0000","last_active":"2024-09-13T09:30:05.434291+0000","last_peered":"2024-09-13T09:30:05.434291+0000","last_clean":"2024-09-13T09:30:05.434291+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:05.434291+0000","last_undegraded":"2024-09-13T09:30:05.434291+0000","last_fullsized":"2024-09-13T09:30:05.434291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:02.710053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":240,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748370+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:30:06.748370+0000","last_peered":"2024-09-13T09:30:06.748370+0000","last_clean":"2024-09-13T09:30:06.748370+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:06.748370+0000","last_undegraded":"2024-09-13T09:30:06.748370+0000","last_fullsized":"2024-09-13T09:30:06.748370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"217'540","reported_seq":2909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748496+0000","last_change":"2024-09-13T09:29:51.341689+0000","last_active":"2024-09-13T09:30:06.748496+0000","last_peered":"2024-09-13T09:30:06.748496+0000","last_clean":"2024-09-13T09:30:06.748496+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:06.748496+0000","last_undegraded":"2024-09-13T09:30:06.748496+0000","last_fullsized":"2024-09-13T09:30:06.748496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:56.843638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":248,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.749025+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:30:06.749025+0000","last_peered":"2024-09-13T09:30:06.749025+0000","last_clean":"2024-09-13T09:30:06.749025+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:06.749025+0000","last_undegraded":"2024-09-13T09:30:06.749025+0000","last_fullsized":"2024-09-13T09:30:06.749025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"210'965","reported_seq":4809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.610219+0000","last_change":"2024-09-13T09:29:44.322446+0000","last_active":"2024-09-13T09:30:06.610219+0000","last_peered":"2024-09-13T09:30:06.610219+0000","last_clean":"2024-09-13T09:30:06.610219+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:06.610219+0000","last_undegraded":"2024-09-13T09:30:06.610219+0000","last_fullsized":"2024-09-13T09:30:06.610219+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:22.045121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434427+0000","last_change":"2024-09-13T09:29:40.201770+0000","last_active":"2024-09-13T09:30:05.434427+0000","last_peered":"2024-09-13T09:30:05.434427+0000","last_clean":"2024-09-13T09:30:05.434427+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:05.434427+0000","last_undegraded":"2024-09-13T09:30:05.434427+0000","last_fullsized":"2024-09-13T09:30:05.434427+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:07.863545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9190,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433166+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:05.433166+0000","last_peered":"2024-09-13T09:30:05.433166+0000","last_clean":"2024-09-13T09:30:05.433166+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:05.433166+0000","last_undegraded":"2024-09-13T09:30:05.433166+0000","last_fullsized":"2024-09-13T09:30:05.433166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":200,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433208+0000","last_change":"2024-09-13T09:29:51.281204+0000","last_active":"2024-09-13T09:30:05.433208+0000","last_peered":"2024-09-13T09:30:05.433208+0000","last_clean":"2024-09-13T09:30:05.433208+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:05.433208+0000","last_undegraded":"2024-09-13T09:30:05.433208+0000","last_fullsized":"2024-09-13T09:30:05.433208+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:33.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'928","reported_seq":6275,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700427+0000","last_change":"2024-09-13T09:29:44.278448+0000","last_active":"2024-09-13T09:30:04.700427+0000","last_peered":"2024-09-13T09:30:04.700427+0000","last_clean":"2024-09-13T09:30:04.700427+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:04.700427+0000","last_undegraded":"2024-09-13T09:30:04.700427+0000","last_fullsized":"2024-09-13T09:30:04.700427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'1424","reported_seq":4477,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699603+0000","last_change":"2024-09-13T09:29:44.275636+0000","last_active":"2024-09-13T09:30:04.699603+0000","last_peered":"2024-09-13T09:30:04.699603+0000","last_clean":"2024-09-13T09:30:04.699603+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:04.699603+0000","last_undegraded":"2024-09-13T09:30:04.699603+0000","last_fullsized":"2024-09-13T09:30:04.699603+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:27.580162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":371,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747540+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:30:05.747540+0000","last_peered":"2024-09-13T09:30:05.747540+0000","last_clean":"2024-09-13T09:30:05.747540+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:05.747540+0000","last_undegraded":"2024-09-13T09:30:05.747540+0000","last_fullsized":"2024-09-13T09:30:05.747540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":229,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246866+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:30:06.246866+0000","last_peered":"2024-09-13T09:30:06.246866+0000","last_clean":"2024-09-13T09:30:06.246866+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:06.246866+0000","last_undegraded":"2024-09-13T09:30:06.246866+0000","last_fullsized":"2024-09-13T09:30:06.246866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9538,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246633+0000","last_change":"2024-09-13T09:29:44.321698+0000","last_active":"2024-09-13T09:30:06.246633+0000","last_peered":"2024-09-13T09:30:06.246633+0000","last_clean":"2024-09-13T09:30:06.246633+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:06.246633+0000","last_undegraded":"2024-09-13T09:30:06.246633+0000","last_fullsized":"2024-09-13T09:30:06.246633+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:11.466841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747653+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:30:05.747653+0000","last_peered":"2024-09-13T09:30:05.747653+0000","last_clean":"2024-09-13T09:30:05.747653+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:05.747653+0000","last_undegraded":"2024-09-13T09:30:05.747653+0000","last_fullsized":"2024-09-13T09:30:05.747653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"212'840","reported_seq":11367,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434421+0000","last_change":"2024-09-13T09:29:44.238109+0000","last_active":"2024-09-13T09:30:05.434421+0000","last_peered":"2024-09-13T09:30:05.434421+0000","last_clean":"2024-09-13T09:30:05.434421+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:05.434421+0000","last_undegraded":"2024-09-13T09:30:05.434421+0000","last_fullsized":"2024-09-13T09:30:05.434421+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.423203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":182,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748225+0000","last_change":"2024-09-13T09:29:51.279392+0000","last_active":"2024-09-13T09:30:06.748225+0000","last_peered":"2024-09-13T09:30:06.748225+0000","last_clean":"2024-09-13T09:30:06.748225+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:06.748225+0000","last_undegraded":"2024-09-13T09:30:06.748225+0000","last_fullsized":"2024-09-13T09:30:06.748225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:37.997623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7101,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432919+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:05.432919+0000","last_peered":"2024-09-13T09:30:05.432919+0000","last_clean":"2024-09-13T09:30:05.432919+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:05.432919+0000","last_undegraded":"2024-09-13T09:30:05.432919+0000","last_fullsized":"2024-09-13T09:30:05.432919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:28:54.042990+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747532+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:30:05.747532+0000","last_peered":"2024-09-13T09:30:05.747532+0000","last_clean":"2024-09-13T09:30:05.747532+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:05.747532+0000","last_undegraded":"2024-09-13T09:30:05.747532+0000","last_fullsized":"2024-09-13T09:30:05.747532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'653","reported_seq":3505,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432879+0000","last_change":"2024-09-13T09:29:44.370982+0000","last_active":"2024-09-13T09:30:05.432879+0000","last_peered":"2024-09-13T09:30:05.432879+0000","last_clean":"2024-09-13T09:30:05.432879+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:05.432879+0000","last_undegraded":"2024-09-13T09:30:05.432879+0000","last_fullsized":"2024-09-13T09:30:05.432879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"209'483","reported_seq":12049,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434041+0000","last_change":"2024-09-13T09:29:44.237484+0000","last_active":"2024-09-13T09:30:05.434041+0000","last_peered":"2024-09-13T09:30:05.434041+0000","last_clean":"2024-09-13T09:30:05.434041+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:05.434041+0000","last_undegraded":"2024-09-13T09:30:05.434041+0000","last_fullsized":"2024-09-13T09:30:05.434041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:37.361009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'572","reported_seq":7649,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748310+0000","last_change":"2024-09-13T09:29:44.229060+0000","last_active":"2024-09-13T09:30:06.748310+0000","last_peered":"2024-09-13T09:30:06.748310+0000","last_clean":"2024-09-13T09:30:06.748310+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:06.748310+0000","last_undegraded":"2024-09-13T09:30:06.748310+0000","last_fullsized":"2024-09-13T09:30:06.748310+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:07.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"210'622","reported_seq":4506,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433036+0000","last_change":"2024-09-13T09:29:44.371528+0000","last_active":"2024-09-13T09:30:05.433036+0000","last_peered":"2024-09-13T09:30:05.433036+0000","last_clean":"2024-09-13T09:30:05.433036+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:05.433036+0000","last_undegraded":"2024-09-13T09:30:05.433036+0000","last_fullsized":"2024-09-13T09:30:05.433036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7640,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748768+0000","last_change":"2024-09-13T09:29:51.357978+0000","last_active":"2024-09-13T09:30:06.748768+0000","last_peered":"2024-09-13T09:30:06.748768+0000","last_clean":"2024-09-13T09:30:06.748768+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:06.748768+0000","last_undegraded":"2024-09-13T09:30:06.748768+0000","last_fullsized":"2024-09-13T09:30:06.748768+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:32.572241+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"217'498","reported_seq":14666,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747792+0000","last_change":"2024-09-13T09:29:51.366598+0000","last_active":"2024-09-13T09:30:05.747792+0000","last_peered":"2024-09-13T09:30:05.747792+0000","last_clean":"2024-09-13T09:30:05.747792+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:05.747792+0000","last_undegraded":"2024-09-13T09:30:05.747792+0000","last_fullsized":"2024-09-13T09:30:05.747792+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:25:28.046650+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"211'1205","reported_seq":9159,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747948+0000","last_change":"2024-09-13T09:29:44.236105+0000","last_active":"2024-09-13T09:30:05.747948+0000","last_peered":"2024-09-13T09:30:05.747948+0000","last_clean":"2024-09-13T09:30:05.747948+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:05.747948+0000","last_undegraded":"2024-09-13T09:30:05.747948+0000","last_fullsized":"2024-09-13T09:30:05.747948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:30.493572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'483","reported_seq":7265,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747872+0000","last_change":"2024-09-13T09:29:44.236466+0000","last_active":"2024-09-13T09:30:05.747872+0000","last_peered":"2024-09-13T09:30:05.747872+0000","last_clean":"2024-09-13T09:30:05.747872+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:05.747872+0000","last_undegraded":"2024-09-13T09:30:05.747872+0000","last_fullsized":"2024-09-13T09:30:05.747872+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:24.511124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'887","reported_seq":4070,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748769+0000","last_change":"2024-09-13T09:29:44.229062+0000","last_active":"2024-09-13T09:30:06.748769+0000","last_peered":"2024-09-13T09:30:06.748769+0000","last_clean":"2024-09-13T09:30:06.748769+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:06.748769+0000","last_undegraded":"2024-09-13T09:30:06.748769+0000","last_fullsized":"2024-09-13T09:30:06.748769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"212'895","reported_seq":9205,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747542+0000","last_change":"2024-09-13T09:29:44.235484+0000","last_active":"2024-09-13T09:30:05.747542+0000","last_peered":"2024-09-13T09:30:05.747542+0000","last_clean":"2024-09-13T09:30:05.747542+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:05.747542+0000","last_undegraded":"2024-09-13T09:30:05.747542+0000","last_fullsized":"2024-09-13T09:30:05.747542+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:42.588638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6241,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699859+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:04.699859+0000","last_peered":"2024-09-13T09:30:04.699859+0000","last_clean":"2024-09-13T09:30:04.699859+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:04.699859+0000","last_undegraded":"2024-09-13T09:30:04.699859+0000","last_fullsized":"2024-09-13T09:30:04.699859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"209'1122","reported_seq":9794,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747717+0000","last_change":"2024-09-13T09:29:44.234845+0000","last_active":"2024-09-13T09:30:05.747717+0000","last_peered":"2024-09-13T09:30:05.747717+0000","last_clean":"2024-09-13T09:30:05.747717+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:05.747717+0000","last_undegraded":"2024-09-13T09:30:05.747717+0000","last_fullsized":"2024-09-13T09:30:05.747717+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:30.144378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25343,"ondisk_log_size":25343,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51976,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.099},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.145}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.1799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509497,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55976,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315864,"statfs":{"total":96636764160,"available":96579444736,"internally_reserved":0,"allocated":2777088,"data_stored":1346707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":1.621}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":1.165},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.393}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":6.9450000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.68799999999999994}]}]},{"osd":5,"up_from":216,"seq":927712935942,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":173,"seq":743029342226,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.712},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.7989999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.534},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1060000000000001}]}]},{"osd":3,"up_from":74,"seq":317827579943,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79924,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291916,"statfs":{"total":96636764160,"available":96554921984,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0529999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.749}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.02},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.099},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.355},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.391}]}]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.246}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.155},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":1.5549999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.286}]}]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.876}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.117},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":0.752},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":1.2250000000000001}]}]},{"osd":2,"up_from":207,"seq":889058230280,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:12.192 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:12.501 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:12.504 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:12.516 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-13T09:30:10.594659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25463,"ondisk_log_size":25463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395168,"kb_used_data":21980,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579552,"statfs":{"total":773094113280,"available":772689461248,"internally_reserved":0,"allocated":22507520,"data_stored":12029970,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.179818"},"pg_stats":[{"pgid":"3.1f","version":"217'535","reported_seq":2244,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700567+0000","last_change":"2024-09-13T09:29:51.329628+0000","last_active":"2024-09-13T09:30:04.700567+0000","last_peered":"2024-09-13T09:30:04.700567+0000","last_clean":"2024-09-13T09:30:04.700567+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:04.700567+0000","last_undegraded":"2024-09-13T09:30:04.700567+0000","last_fullsized":"2024-09-13T09:30:04.700567+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:43.571566+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"209'831","reported_seq":5831,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433026+0000","last_change":"2024-09-13T09:29:44.372628+0000","last_active":"2024-09-13T09:30:05.433026+0000","last_peered":"2024-09-13T09:30:05.433026+0000","last_clean":"2024-09-13T09:30:05.433026+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:05.433026+0000","last_undegraded":"2024-09-13T09:30:05.433026+0000","last_fullsized":"2024-09-13T09:30:05.433026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'629","reported_seq":5438,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748750+0000","last_change":"2024-09-13T09:29:52.302657+0000","last_active":"2024-09-13T09:30:06.748750+0000","last_peered":"2024-09-13T09:30:06.748750+0000","last_clean":"2024-09-13T09:30:06.748750+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:06.748750+0000","last_undegraded":"2024-09-13T09:30:06.748750+0000","last_fullsized":"2024-09-13T09:30:06.748750+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:50.114832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'819","reported_seq":6150,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748562+0000","last_change":"2024-09-13T09:29:52.330059+0000","last_active":"2024-09-13T09:30:06.748562+0000","last_peered":"2024-09-13T09:30:06.748562+0000","last_clean":"2024-09-13T09:30:06.748562+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:06.748562+0000","last_undegraded":"2024-09-13T09:30:06.748562+0000","last_fullsized":"2024-09-13T09:30:06.748562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:38.645385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7169,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700079+0000","last_change":"2024-09-13T09:29:44.278432+0000","last_active":"2024-09-13T09:30:04.700079+0000","last_peered":"2024-09-13T09:30:04.700079+0000","last_clean":"2024-09-13T09:30:04.700079+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:04.700079+0000","last_undegraded":"2024-09-13T09:30:04.700079+0000","last_fullsized":"2024-09-13T09:30:04.700079+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:24.917547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'680","reported_seq":13797,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434186+0000","last_change":"2024-09-13T09:29:44.237452+0000","last_active":"2024-09-13T09:30:05.434186+0000","last_peered":"2024-09-13T09:30:05.434186+0000","last_clean":"2024-09-13T09:30:05.434186+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:05.434186+0000","last_undegraded":"2024-09-13T09:30:05.434186+0000","last_fullsized":"2024-09-13T09:30:05.434186+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:22.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'666","reported_seq":3868,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246820+0000","last_change":"2024-09-13T09:29:44.322458+0000","last_active":"2024-09-13T09:30:06.246820+0000","last_peered":"2024-09-13T09:30:06.246820+0000","last_clean":"2024-09-13T09:30:06.246820+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:06.246820+0000","last_undegraded":"2024-09-13T09:30:06.246820+0000","last_fullsized":"2024-09-13T09:30:06.246820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"212'691","reported_seq":5866,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.746774+0000","last_change":"2024-09-13T09:29:51.341364+0000","last_active":"2024-09-13T09:30:06.746774+0000","last_peered":"2024-09-13T09:30:06.746774+0000","last_clean":"2024-09-13T09:30:06.746774+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:06.746774+0000","last_undegraded":"2024-09-13T09:30:06.746774+0000","last_fullsized":"2024-09-13T09:30:06.746774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:42.266961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"211'788","reported_seq":3015,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700432+0000","last_change":"2024-09-13T09:29:54.325150+0000","last_active":"2024-09-13T09:30:04.700432+0000","last_peered":"2024-09-13T09:30:04.700432+0000","last_clean":"2024-09-13T09:30:04.700432+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:04.700432+0000","last_undegraded":"2024-09-13T09:30:04.700432+0000","last_fullsized":"2024-09-13T09:30:04.700432+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:25.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6166,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748498+0000","last_change":"2024-09-13T09:29:51.321889+0000","last_active":"2024-09-13T09:30:06.748498+0000","last_peered":"2024-09-13T09:30:06.748498+0000","last_clean":"2024-09-13T09:30:06.748498+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:06.748498+0000","last_undegraded":"2024-09-13T09:30:06.748498+0000","last_fullsized":"2024-09-13T09:30:06.748498+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:12.873717+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'497","reported_seq":10587,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747787+0000","last_change":"2024-09-13T09:29:44.233280+0000","last_active":"2024-09-13T09:30:05.747787+0000","last_peered":"2024-09-13T09:30:05.747787+0000","last_clean":"2024-09-13T09:30:05.747787+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:05.747787+0000","last_undegraded":"2024-09-13T09:30:05.747787+0000","last_fullsized":"2024-09-13T09:30:05.747787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"210'613","reported_seq":6519,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434291+0000","last_change":"2024-09-13T09:29:44.238073+0000","last_active":"2024-09-13T09:30:05.434291+0000","last_peered":"2024-09-13T09:30:05.434291+0000","last_clean":"2024-09-13T09:30:05.434291+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:05.434291+0000","last_undegraded":"2024-09-13T09:30:05.434291+0000","last_fullsized":"2024-09-13T09:30:05.434291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:02.710053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":240,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748370+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:30:06.748370+0000","last_peered":"2024-09-13T09:30:06.748370+0000","last_clean":"2024-09-13T09:30:06.748370+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:06.748370+0000","last_undegraded":"2024-09-13T09:30:06.748370+0000","last_fullsized":"2024-09-13T09:30:06.748370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"217'540","reported_seq":2909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748496+0000","last_change":"2024-09-13T09:29:51.341689+0000","last_active":"2024-09-13T09:30:06.748496+0000","last_peered":"2024-09-13T09:30:06.748496+0000","last_clean":"2024-09-13T09:30:06.748496+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:06.748496+0000","last_undegraded":"2024-09-13T09:30:06.748496+0000","last_fullsized":"2024-09-13T09:30:06.748496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:56.843638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":248,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.749025+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:30:06.749025+0000","last_peered":"2024-09-13T09:30:06.749025+0000","last_clean":"2024-09-13T09:30:06.749025+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:06.749025+0000","last_undegraded":"2024-09-13T09:30:06.749025+0000","last_fullsized":"2024-09-13T09:30:06.749025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"210'965","reported_seq":4809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.610219+0000","last_change":"2024-09-13T09:29:44.322446+0000","last_active":"2024-09-13T09:30:06.610219+0000","last_peered":"2024-09-13T09:30:06.610219+0000","last_clean":"2024-09-13T09:30:06.610219+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:06.610219+0000","last_undegraded":"2024-09-13T09:30:06.610219+0000","last_fullsized":"2024-09-13T09:30:06.610219+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:22.045121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434427+0000","last_change":"2024-09-13T09:29:40.201770+0000","last_active":"2024-09-13T09:30:05.434427+0000","last_peered":"2024-09-13T09:30:05.434427+0000","last_clean":"2024-09-13T09:30:05.434427+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:05.434427+0000","last_undegraded":"2024-09-13T09:30:05.434427+0000","last_fullsized":"2024-09-13T09:30:05.434427+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:07.863545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9190,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433166+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:05.433166+0000","last_peered":"2024-09-13T09:30:05.433166+0000","last_clean":"2024-09-13T09:30:05.433166+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:05.433166+0000","last_undegraded":"2024-09-13T09:30:05.433166+0000","last_fullsized":"2024-09-13T09:30:05.433166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":200,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433208+0000","last_change":"2024-09-13T09:29:51.281204+0000","last_active":"2024-09-13T09:30:05.433208+0000","last_peered":"2024-09-13T09:30:05.433208+0000","last_clean":"2024-09-13T09:30:05.433208+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:05.433208+0000","last_undegraded":"2024-09-13T09:30:05.433208+0000","last_fullsized":"2024-09-13T09:30:05.433208+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:33.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'928","reported_seq":6275,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700427+0000","last_change":"2024-09-13T09:29:44.278448+0000","last_active":"2024-09-13T09:30:04.700427+0000","last_peered":"2024-09-13T09:30:04.700427+0000","last_clean":"2024-09-13T09:30:04.700427+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:04.700427+0000","last_undegraded":"2024-09-13T09:30:04.700427+0000","last_fullsized":"2024-09-13T09:30:04.700427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'1424","reported_seq":4477,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699603+0000","last_change":"2024-09-13T09:29:44.275636+0000","last_active":"2024-09-13T09:30:04.699603+0000","last_peered":"2024-09-13T09:30:04.699603+0000","last_clean":"2024-09-13T09:30:04.699603+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:04.699603+0000","last_undegraded":"2024-09-13T09:30:04.699603+0000","last_fullsized":"2024-09-13T09:30:04.699603+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:27.580162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":371,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747540+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:30:05.747540+0000","last_peered":"2024-09-13T09:30:05.747540+0000","last_clean":"2024-09-13T09:30:05.747540+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:05.747540+0000","last_undegraded":"2024-09-13T09:30:05.747540+0000","last_fullsized":"2024-09-13T09:30:05.747540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":229,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246866+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:30:06.246866+0000","last_peered":"2024-09-13T09:30:06.246866+0000","last_clean":"2024-09-13T09:30:06.246866+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:06.246866+0000","last_undegraded":"2024-09-13T09:30:06.246866+0000","last_fullsized":"2024-09-13T09:30:06.246866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9538,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246633+0000","last_change":"2024-09-13T09:29:44.321698+0000","last_active":"2024-09-13T09:30:06.246633+0000","last_peered":"2024-09-13T09:30:06.246633+0000","last_clean":"2024-09-13T09:30:06.246633+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:06.246633+0000","last_undegraded":"2024-09-13T09:30:06.246633+0000","last_fullsized":"2024-09-13T09:30:06.246633+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:11.466841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747653+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:30:05.747653+0000","last_peered":"2024-09-13T09:30:05.747653+0000","last_clean":"2024-09-13T09:30:05.747653+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:05.747653+0000","last_undegraded":"2024-09-13T09:30:05.747653+0000","last_fullsized":"2024-09-13T09:30:05.747653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"212'840","reported_seq":11367,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434421+0000","last_change":"2024-09-13T09:29:44.238109+0000","last_active":"2024-09-13T09:30:05.434421+0000","last_peered":"2024-09-13T09:30:05.434421+0000","last_clean":"2024-09-13T09:30:05.434421+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:05.434421+0000","last_undegraded":"2024-09-13T09:30:05.434421+0000","last_fullsized":"2024-09-13T09:30:05.434421+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.423203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":182,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748225+0000","last_change":"2024-09-13T09:29:51.279392+0000","last_active":"2024-09-13T09:30:06.748225+0000","last_peered":"2024-09-13T09:30:06.748225+0000","last_clean":"2024-09-13T09:30:06.748225+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:06.748225+0000","last_undegraded":"2024-09-13T09:30:06.748225+0000","last_fullsized":"2024-09-13T09:30:06.748225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:37.997623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7101,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432919+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:05.432919+0000","last_peered":"2024-09-13T09:30:05.432919+0000","last_clean":"2024-09-13T09:30:05.432919+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:05.432919+0000","last_undegraded":"2024-09-13T09:30:05.432919+0000","last_fullsized":"2024-09-13T09:30:05.432919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:28:54.042990+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747532+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:30:05.747532+0000","last_peered":"2024-09-13T09:30:05.747532+0000","last_clean":"2024-09-13T09:30:05.747532+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:05.747532+0000","last_undegraded":"2024-09-13T09:30:05.747532+0000","last_fullsized":"2024-09-13T09:30:05.747532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'653","reported_seq":3505,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432879+0000","last_change":"2024-09-13T09:29:44.370982+0000","last_active":"2024-09-13T09:30:05.432879+0000","last_peered":"2024-09-13T09:30:05.432879+0000","last_clean":"2024-09-13T09:30:05.432879+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:05.432879+0000","last_undegraded":"2024-09-13T09:30:05.432879+0000","last_fullsized":"2024-09-13T09:30:05.432879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"209'483","reported_seq":12049,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434041+0000","last_change":"2024-09-13T09:29:44.237484+0000","last_active":"2024-09-13T09:30:05.434041+0000","last_peered":"2024-09-13T09:30:05.434041+0000","last_clean":"2024-09-13T09:30:05.434041+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:05.434041+0000","last_undegraded":"2024-09-13T09:30:05.434041+0000","last_fullsized":"2024-09-13T09:30:05.434041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:37.361009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'572","reported_seq":7649,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748310+0000","last_change":"2024-09-13T09:29:44.229060+0000","last_active":"2024-09-13T09:30:06.748310+0000","last_peered":"2024-09-13T09:30:06.748310+0000","last_clean":"2024-09-13T09:30:06.748310+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:06.748310+0000","last_undegraded":"2024-09-13T09:30:06.748310+0000","last_fullsized":"2024-09-13T09:30:06.748310+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:07.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"210'622","reported_seq":4506,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433036+0000","last_change":"2024-09-13T09:29:44.371528+0000","last_active":"2024-09-13T09:30:05.433036+0000","last_peered":"2024-09-13T09:30:05.433036+0000","last_clean":"2024-09-13T09:30:05.433036+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:05.433036+0000","last_undegraded":"2024-09-13T09:30:05.433036+0000","last_fullsized":"2024-09-13T09:30:05.433036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7640,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748768+0000","last_change":"2024-09-13T09:29:51.357978+0000","last_active":"2024-09-13T09:30:06.748768+0000","last_peered":"2024-09-13T09:30:06.748768+0000","last_clean":"2024-09-13T09:30:06.748768+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:06.748768+0000","last_undegraded":"2024-09-13T09:30:06.748768+0000","last_fullsized":"2024-09-13T09:30:06.748768+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:32.572241+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"217'498","reported_seq":14666,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747792+0000","last_change":"2024-09-13T09:29:51.366598+0000","last_active":"2024-09-13T09:30:05.747792+0000","last_peered":"2024-09-13T09:30:05.747792+0000","last_clean":"2024-09-13T09:30:05.747792+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:05.747792+0000","last_undegraded":"2024-09-13T09:30:05.747792+0000","last_fullsized":"2024-09-13T09:30:05.747792+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:25:28.046650+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"211'1205","reported_seq":9159,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747948+0000","last_change":"2024-09-13T09:29:44.236105+0000","last_active":"2024-09-13T09:30:05.747948+0000","last_peered":"2024-09-13T09:30:05.747948+0000","last_clean":"2024-09-13T09:30:05.747948+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:05.747948+0000","last_undegraded":"2024-09-13T09:30:05.747948+0000","last_fullsized":"2024-09-13T09:30:05.747948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:30.493572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'483","reported_seq":7265,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747872+0000","last_change":"2024-09-13T09:29:44.236466+0000","last_active":"2024-09-13T09:30:05.747872+0000","last_peered":"2024-09-13T09:30:05.747872+0000","last_clean":"2024-09-13T09:30:05.747872+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:05.747872+0000","last_undegraded":"2024-09-13T09:30:05.747872+0000","last_fullsized":"2024-09-13T09:30:05.747872+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:24.511124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'887","reported_seq":4070,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748769+0000","last_change":"2024-09-13T09:29:44.229062+0000","last_active":"2024-09-13T09:30:06.748769+0000","last_peered":"2024-09-13T09:30:06.748769+0000","last_clean":"2024-09-13T09:30:06.748769+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:06.748769+0000","last_undegraded":"2024-09-13T09:30:06.748769+0000","last_fullsized":"2024-09-13T09:30:06.748769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"212'895","reported_seq":9205,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747542+0000","last_change":"2024-09-13T09:29:44.235484+0000","last_active":"2024-09-13T09:30:05.747542+0000","last_peered":"2024-09-13T09:30:05.747542+0000","last_clean":"2024-09-13T09:30:05.747542+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:05.747542+0000","last_undegraded":"2024-09-13T09:30:05.747542+0000","last_fullsized":"2024-09-13T09:30:05.747542+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:42.588638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6241,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699859+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:04.699859+0000","last_peered":"2024-09-13T09:30:04.699859+0000","last_clean":"2024-09-13T09:30:04.699859+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:04.699859+0000","last_undegraded":"2024-09-13T09:30:04.699859+0000","last_fullsized":"2024-09-13T09:30:04.699859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"209'1122","reported_seq":9794,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747717+0000","last_change":"2024-09-13T09:29:44.234845+0000","last_active":"2024-09-13T09:30:05.747717+0000","last_peered":"2024-09-13T09:30:05.747717+0000","last_clean":"2024-09-13T09:30:05.747717+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:05.747717+0000","last_undegraded":"2024-09-13T09:30:05.747717+0000","last_fullsized":"2024-09-13T09:30:05.747717+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:30.144378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25343,"ondisk_log_size":25343,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51976,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.099},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.751}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.145}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.2110000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.1799999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509497,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55976,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315864,"statfs":{"total":96636764160,"available":96579444736,"internally_reserved":0,"allocated":2777088,"data_stored":1346707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":1.621}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":1.165},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.393}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":6.9450000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.54200000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.68799999999999994}]}]},{"osd":5,"up_from":216,"seq":927712935942,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":173,"seq":743029342226,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.712},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.7989999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.534},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1060000000000001}]}]},{"osd":3,"up_from":74,"seq":317827579943,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79924,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291916,"statfs":{"total":96636764160,"available":96554921984,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.2569999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0529999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.749}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.02},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.099},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.4490000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.3049999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.355},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.391}]}]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.246}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.4279999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.365}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.155},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":1.5549999999999999},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.286}]}]},{"osd":0,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":0.876}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.117},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":0.752},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":1.2649999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":1.2250000000000001}]}]},{"osd":2,"up_from":207,"seq":889058230280,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:12.519 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:30:12.519 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:12.833 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:12.837 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:12.850 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-13T09:30:12.595253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25463,"ondisk_log_size":25463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395192,"kb_used_data":22004,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579528,"statfs":{"total":773094113280,"available":772689436672,"internally_reserved":0,"allocated":22532096,"data_stored":12041654,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002533"},"pg_stats":[{"pgid":"3.1f","version":"217'535","reported_seq":2244,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700567+0000","last_change":"2024-09-13T09:29:51.329628+0000","last_active":"2024-09-13T09:30:04.700567+0000","last_peered":"2024-09-13T09:30:04.700567+0000","last_clean":"2024-09-13T09:30:04.700567+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:04.700567+0000","last_undegraded":"2024-09-13T09:30:04.700567+0000","last_fullsized":"2024-09-13T09:30:04.700567+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:43.571566+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"209'831","reported_seq":5831,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433026+0000","last_change":"2024-09-13T09:29:44.372628+0000","last_active":"2024-09-13T09:30:05.433026+0000","last_peered":"2024-09-13T09:30:05.433026+0000","last_clean":"2024-09-13T09:30:05.433026+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:05.433026+0000","last_undegraded":"2024-09-13T09:30:05.433026+0000","last_fullsized":"2024-09-13T09:30:05.433026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'629","reported_seq":5438,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748750+0000","last_change":"2024-09-13T09:29:52.302657+0000","last_active":"2024-09-13T09:30:06.748750+0000","last_peered":"2024-09-13T09:30:06.748750+0000","last_clean":"2024-09-13T09:30:06.748750+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:06.748750+0000","last_undegraded":"2024-09-13T09:30:06.748750+0000","last_fullsized":"2024-09-13T09:30:06.748750+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:50.114832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'819","reported_seq":6150,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748562+0000","last_change":"2024-09-13T09:29:52.330059+0000","last_active":"2024-09-13T09:30:06.748562+0000","last_peered":"2024-09-13T09:30:06.748562+0000","last_clean":"2024-09-13T09:30:06.748562+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:06.748562+0000","last_undegraded":"2024-09-13T09:30:06.748562+0000","last_fullsized":"2024-09-13T09:30:06.748562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:38.645385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7169,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700079+0000","last_change":"2024-09-13T09:29:44.278432+0000","last_active":"2024-09-13T09:30:04.700079+0000","last_peered":"2024-09-13T09:30:04.700079+0000","last_clean":"2024-09-13T09:30:04.700079+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:04.700079+0000","last_undegraded":"2024-09-13T09:30:04.700079+0000","last_fullsized":"2024-09-13T09:30:04.700079+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:24.917547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'680","reported_seq":13797,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434186+0000","last_change":"2024-09-13T09:29:44.237452+0000","last_active":"2024-09-13T09:30:05.434186+0000","last_peered":"2024-09-13T09:30:05.434186+0000","last_clean":"2024-09-13T09:30:05.434186+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:05.434186+0000","last_undegraded":"2024-09-13T09:30:05.434186+0000","last_fullsized":"2024-09-13T09:30:05.434186+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:22.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'666","reported_seq":3868,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246820+0000","last_change":"2024-09-13T09:29:44.322458+0000","last_active":"2024-09-13T09:30:06.246820+0000","last_peered":"2024-09-13T09:30:06.246820+0000","last_clean":"2024-09-13T09:30:06.246820+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:06.246820+0000","last_undegraded":"2024-09-13T09:30:06.246820+0000","last_fullsized":"2024-09-13T09:30:06.246820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"212'691","reported_seq":5866,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.746774+0000","last_change":"2024-09-13T09:29:51.341364+0000","last_active":"2024-09-13T09:30:06.746774+0000","last_peered":"2024-09-13T09:30:06.746774+0000","last_clean":"2024-09-13T09:30:06.746774+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:06.746774+0000","last_undegraded":"2024-09-13T09:30:06.746774+0000","last_fullsized":"2024-09-13T09:30:06.746774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:42.266961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"211'788","reported_seq":3015,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700432+0000","last_change":"2024-09-13T09:29:54.325150+0000","last_active":"2024-09-13T09:30:04.700432+0000","last_peered":"2024-09-13T09:30:04.700432+0000","last_clean":"2024-09-13T09:30:04.700432+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:04.700432+0000","last_undegraded":"2024-09-13T09:30:04.700432+0000","last_fullsized":"2024-09-13T09:30:04.700432+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:25.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6166,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748498+0000","last_change":"2024-09-13T09:29:51.321889+0000","last_active":"2024-09-13T09:30:06.748498+0000","last_peered":"2024-09-13T09:30:06.748498+0000","last_clean":"2024-09-13T09:30:06.748498+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:06.748498+0000","last_undegraded":"2024-09-13T09:30:06.748498+0000","last_fullsized":"2024-09-13T09:30:06.748498+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:12.873717+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'497","reported_seq":10587,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747787+0000","last_change":"2024-09-13T09:29:44.233280+0000","last_active":"2024-09-13T09:30:05.747787+0000","last_peered":"2024-09-13T09:30:05.747787+0000","last_clean":"2024-09-13T09:30:05.747787+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:05.747787+0000","last_undegraded":"2024-09-13T09:30:05.747787+0000","last_fullsized":"2024-09-13T09:30:05.747787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"210'613","reported_seq":6519,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434291+0000","last_change":"2024-09-13T09:29:44.238073+0000","last_active":"2024-09-13T09:30:05.434291+0000","last_peered":"2024-09-13T09:30:05.434291+0000","last_clean":"2024-09-13T09:30:05.434291+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:05.434291+0000","last_undegraded":"2024-09-13T09:30:05.434291+0000","last_fullsized":"2024-09-13T09:30:05.434291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:02.710053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":240,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748370+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:30:06.748370+0000","last_peered":"2024-09-13T09:30:06.748370+0000","last_clean":"2024-09-13T09:30:06.748370+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:06.748370+0000","last_undegraded":"2024-09-13T09:30:06.748370+0000","last_fullsized":"2024-09-13T09:30:06.748370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"217'540","reported_seq":2909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748496+0000","last_change":"2024-09-13T09:29:51.341689+0000","last_active":"2024-09-13T09:30:06.748496+0000","last_peered":"2024-09-13T09:30:06.748496+0000","last_clean":"2024-09-13T09:30:06.748496+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:06.748496+0000","last_undegraded":"2024-09-13T09:30:06.748496+0000","last_fullsized":"2024-09-13T09:30:06.748496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:56.843638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":248,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.749025+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:30:06.749025+0000","last_peered":"2024-09-13T09:30:06.749025+0000","last_clean":"2024-09-13T09:30:06.749025+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:06.749025+0000","last_undegraded":"2024-09-13T09:30:06.749025+0000","last_fullsized":"2024-09-13T09:30:06.749025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"210'965","reported_seq":4809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.610219+0000","last_change":"2024-09-13T09:29:44.322446+0000","last_active":"2024-09-13T09:30:06.610219+0000","last_peered":"2024-09-13T09:30:06.610219+0000","last_clean":"2024-09-13T09:30:06.610219+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:06.610219+0000","last_undegraded":"2024-09-13T09:30:06.610219+0000","last_fullsized":"2024-09-13T09:30:06.610219+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:22.045121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434427+0000","last_change":"2024-09-13T09:29:40.201770+0000","last_active":"2024-09-13T09:30:05.434427+0000","last_peered":"2024-09-13T09:30:05.434427+0000","last_clean":"2024-09-13T09:30:05.434427+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:05.434427+0000","last_undegraded":"2024-09-13T09:30:05.434427+0000","last_fullsized":"2024-09-13T09:30:05.434427+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:07.863545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9190,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433166+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:05.433166+0000","last_peered":"2024-09-13T09:30:05.433166+0000","last_clean":"2024-09-13T09:30:05.433166+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:05.433166+0000","last_undegraded":"2024-09-13T09:30:05.433166+0000","last_fullsized":"2024-09-13T09:30:05.433166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":200,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433208+0000","last_change":"2024-09-13T09:29:51.281204+0000","last_active":"2024-09-13T09:30:05.433208+0000","last_peered":"2024-09-13T09:30:05.433208+0000","last_clean":"2024-09-13T09:30:05.433208+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:05.433208+0000","last_undegraded":"2024-09-13T09:30:05.433208+0000","last_fullsized":"2024-09-13T09:30:05.433208+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:33.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'928","reported_seq":6275,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700427+0000","last_change":"2024-09-13T09:29:44.278448+0000","last_active":"2024-09-13T09:30:04.700427+0000","last_peered":"2024-09-13T09:30:04.700427+0000","last_clean":"2024-09-13T09:30:04.700427+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:04.700427+0000","last_undegraded":"2024-09-13T09:30:04.700427+0000","last_fullsized":"2024-09-13T09:30:04.700427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'1424","reported_seq":4477,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699603+0000","last_change":"2024-09-13T09:29:44.275636+0000","last_active":"2024-09-13T09:30:04.699603+0000","last_peered":"2024-09-13T09:30:04.699603+0000","last_clean":"2024-09-13T09:30:04.699603+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:04.699603+0000","last_undegraded":"2024-09-13T09:30:04.699603+0000","last_fullsized":"2024-09-13T09:30:04.699603+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:27.580162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":371,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747540+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:30:05.747540+0000","last_peered":"2024-09-13T09:30:05.747540+0000","last_clean":"2024-09-13T09:30:05.747540+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:05.747540+0000","last_undegraded":"2024-09-13T09:30:05.747540+0000","last_fullsized":"2024-09-13T09:30:05.747540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":229,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246866+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:30:06.246866+0000","last_peered":"2024-09-13T09:30:06.246866+0000","last_clean":"2024-09-13T09:30:06.246866+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:06.246866+0000","last_undegraded":"2024-09-13T09:30:06.246866+0000","last_fullsized":"2024-09-13T09:30:06.246866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9538,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246633+0000","last_change":"2024-09-13T09:29:44.321698+0000","last_active":"2024-09-13T09:30:06.246633+0000","last_peered":"2024-09-13T09:30:06.246633+0000","last_clean":"2024-09-13T09:30:06.246633+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:06.246633+0000","last_undegraded":"2024-09-13T09:30:06.246633+0000","last_fullsized":"2024-09-13T09:30:06.246633+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:11.466841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747653+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:30:05.747653+0000","last_peered":"2024-09-13T09:30:05.747653+0000","last_clean":"2024-09-13T09:30:05.747653+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:05.747653+0000","last_undegraded":"2024-09-13T09:30:05.747653+0000","last_fullsized":"2024-09-13T09:30:05.747653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"212'840","reported_seq":11367,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434421+0000","last_change":"2024-09-13T09:29:44.238109+0000","last_active":"2024-09-13T09:30:05.434421+0000","last_peered":"2024-09-13T09:30:05.434421+0000","last_clean":"2024-09-13T09:30:05.434421+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:05.434421+0000","last_undegraded":"2024-09-13T09:30:05.434421+0000","last_fullsized":"2024-09-13T09:30:05.434421+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.423203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":182,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748225+0000","last_change":"2024-09-13T09:29:51.279392+0000","last_active":"2024-09-13T09:30:06.748225+0000","last_peered":"2024-09-13T09:30:06.748225+0000","last_clean":"2024-09-13T09:30:06.748225+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:06.748225+0000","last_undegraded":"2024-09-13T09:30:06.748225+0000","last_fullsized":"2024-09-13T09:30:06.748225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:37.997623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7101,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432919+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:05.432919+0000","last_peered":"2024-09-13T09:30:05.432919+0000","last_clean":"2024-09-13T09:30:05.432919+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:05.432919+0000","last_undegraded":"2024-09-13T09:30:05.432919+0000","last_fullsized":"2024-09-13T09:30:05.432919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:28:54.042990+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747532+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:30:05.747532+0000","last_peered":"2024-09-13T09:30:05.747532+0000","last_clean":"2024-09-13T09:30:05.747532+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:05.747532+0000","last_undegraded":"2024-09-13T09:30:05.747532+0000","last_fullsized":"2024-09-13T09:30:05.747532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'653","reported_seq":3505,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432879+0000","last_change":"2024-09-13T09:29:44.370982+0000","last_active":"2024-09-13T09:30:05.432879+0000","last_peered":"2024-09-13T09:30:05.432879+0000","last_clean":"2024-09-13T09:30:05.432879+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:05.432879+0000","last_undegraded":"2024-09-13T09:30:05.432879+0000","last_fullsized":"2024-09-13T09:30:05.432879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"209'483","reported_seq":12049,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434041+0000","last_change":"2024-09-13T09:29:44.237484+0000","last_active":"2024-09-13T09:30:05.434041+0000","last_peered":"2024-09-13T09:30:05.434041+0000","last_clean":"2024-09-13T09:30:05.434041+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:05.434041+0000","last_undegraded":"2024-09-13T09:30:05.434041+0000","last_fullsized":"2024-09-13T09:30:05.434041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:37.361009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'572","reported_seq":7649,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748310+0000","last_change":"2024-09-13T09:29:44.229060+0000","last_active":"2024-09-13T09:30:06.748310+0000","last_peered":"2024-09-13T09:30:06.748310+0000","last_clean":"2024-09-13T09:30:06.748310+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:06.748310+0000","last_undegraded":"2024-09-13T09:30:06.748310+0000","last_fullsized":"2024-09-13T09:30:06.748310+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:07.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"210'622","reported_seq":4506,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433036+0000","last_change":"2024-09-13T09:29:44.371528+0000","last_active":"2024-09-13T09:30:05.433036+0000","last_peered":"2024-09-13T09:30:05.433036+0000","last_clean":"2024-09-13T09:30:05.433036+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:05.433036+0000","last_undegraded":"2024-09-13T09:30:05.433036+0000","last_fullsized":"2024-09-13T09:30:05.433036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7640,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748768+0000","last_change":"2024-09-13T09:29:51.357978+0000","last_active":"2024-09-13T09:30:06.748768+0000","last_peered":"2024-09-13T09:30:06.748768+0000","last_clean":"2024-09-13T09:30:06.748768+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:06.748768+0000","last_undegraded":"2024-09-13T09:30:06.748768+0000","last_fullsized":"2024-09-13T09:30:06.748768+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:32.572241+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"217'498","reported_seq":14666,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747792+0000","last_change":"2024-09-13T09:29:51.366598+0000","last_active":"2024-09-13T09:30:05.747792+0000","last_peered":"2024-09-13T09:30:05.747792+0000","last_clean":"2024-09-13T09:30:05.747792+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:05.747792+0000","last_undegraded":"2024-09-13T09:30:05.747792+0000","last_fullsized":"2024-09-13T09:30:05.747792+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:25:28.046650+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"211'1205","reported_seq":9159,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747948+0000","last_change":"2024-09-13T09:29:44.236105+0000","last_active":"2024-09-13T09:30:05.747948+0000","last_peered":"2024-09-13T09:30:05.747948+0000","last_clean":"2024-09-13T09:30:05.747948+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:05.747948+0000","last_undegraded":"2024-09-13T09:30:05.747948+0000","last_fullsized":"2024-09-13T09:30:05.747948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:30.493572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'483","reported_seq":7265,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747872+0000","last_change":"2024-09-13T09:29:44.236466+0000","last_active":"2024-09-13T09:30:05.747872+0000","last_peered":"2024-09-13T09:30:05.747872+0000","last_clean":"2024-09-13T09:30:05.747872+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:05.747872+0000","last_undegraded":"2024-09-13T09:30:05.747872+0000","last_fullsized":"2024-09-13T09:30:05.747872+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:24.511124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'887","reported_seq":4070,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748769+0000","last_change":"2024-09-13T09:29:44.229062+0000","last_active":"2024-09-13T09:30:06.748769+0000","last_peered":"2024-09-13T09:30:06.748769+0000","last_clean":"2024-09-13T09:30:06.748769+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:06.748769+0000","last_undegraded":"2024-09-13T09:30:06.748769+0000","last_fullsized":"2024-09-13T09:30:06.748769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"212'895","reported_seq":9205,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747542+0000","last_change":"2024-09-13T09:29:44.235484+0000","last_active":"2024-09-13T09:30:05.747542+0000","last_peered":"2024-09-13T09:30:05.747542+0000","last_clean":"2024-09-13T09:30:05.747542+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:05.747542+0000","last_undegraded":"2024-09-13T09:30:05.747542+0000","last_fullsized":"2024-09-13T09:30:05.747542+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:42.588638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6241,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699859+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:04.699859+0000","last_peered":"2024-09-13T09:30:04.699859+0000","last_clean":"2024-09-13T09:30:04.699859+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:04.699859+0000","last_undegraded":"2024-09-13T09:30:04.699859+0000","last_fullsized":"2024-09-13T09:30:04.699859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"209'1122","reported_seq":9794,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747717+0000","last_change":"2024-09-13T09:29:44.234845+0000","last_active":"2024-09-13T09:30:05.747717+0000","last_peered":"2024-09-13T09:30:05.747717+0000","last_clean":"2024-09-13T09:30:05.747717+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:05.747717+0000","last_undegraded":"2024-09-13T09:30:05.747717+0000","last_fullsized":"2024-09-13T09:30:05.747717+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:30.144378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25343,"ondisk_log_size":25343,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51976,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.0369999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.626},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.86199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.89900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.079}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.123}]}]},{"osd":6,"up_from":15,"seq":64424509498,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56000,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315840,"statfs":{"total":96636764160,"available":96579420160,"internally_reserved":0,"allocated":2801664,"data_stored":1358391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.127}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.048},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.99399999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.0760000000000001}]}]},{"osd":5,"up_from":216,"seq":927712935942,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.175}]}]},{"osd":4,"up_from":173,"seq":743029342226,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.712},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.5940000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.7989999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.534},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1060000000000001}]}]},{"osd":3,"up_from":74,"seq":317827579944,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79924,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291916,"statfs":{"total":96636764160,"available":96554921984,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.33800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.82599999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.93100000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.432}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.68899999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.502},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.753},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.0029999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.026},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":0.77900000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":0.878}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":1.0649999999999999}]}]},{"osd":2,"up_from":207,"seq":889058230280,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0150000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1019999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:12.852 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:30:13.161 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:13.162 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":232,"fsid":"eb7f03c3-9691-4766-8999-6f4f526cda71","created":"2024-09-13T09:25:30.379173+0000","modified":"2024-09-13T09:30:06.428477+0000","last_up_change":"2024-09-13T09:29:50.256487+0000","last_in_change":"2024-09-13T09:29:08.613117+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":9,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T09:25:41.357433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T09:25:42.578616+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:25:51.228779+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":212,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"54ea2bd8-55b0-4ded-8a16-041261c5ba5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":219,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6817","nonce":2906107514}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6819","nonce":2906107514}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6823","nonce":2906107514}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2906107514},{"type":"v1","addr":"172.21.15.63:6821","nonce":2906107514}]},"public_addr":"172.21.15.63:6817/2906107514","cluster_addr":"172.21.15.63:6819/2906107514","heartbeat_back_addr":"172.21.15.63:6823/2906107514","heartbeat_front_addr":"172.21.15.63:6821/2906107514","state":["exists","up"]},{"osd":1,"uuid":"0c95aee9-f871-4f6f-bbbd-1206db5d331c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6825","nonce":3622854518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6827","nonce":3622854518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6831","nonce":3622854518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":3622854518},{"type":"v1","addr":"172.21.15.63:6829","nonce":3622854518}]},"public_addr":"172.21.15.63:6825/3622854518","cluster_addr":"172.21.15.63:6827/3622854518","heartbeat_back_addr":"172.21.15.63:6831/3622854518","heartbeat_front_addr":"172.21.15.63:6829/3622854518","state":["exists","up"]},{"osd":2,"uuid":"f8253db2-d4da-46b5-95cf-cd5c9ebd7328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":194,"up_from":207,"up_thru":216,"down_at":196,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6801","nonce":2636060942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6803","nonce":2636060942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6807","nonce":2636060942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2636060942},{"type":"v1","addr":"172.21.15.63:6805","nonce":2636060942}]},"public_addr":"172.21.15.63:6801/2636060942","cluster_addr":"172.21.15.63:6803/2636060942","heartbeat_back_addr":"172.21.15.63:6807/2636060942","heartbeat_front_addr":"172.21.15.63:6805/2636060942","state":["exists","up"]},{"osd":3,"uuid":"67c11a52-baeb-4ea8-89ea-26018ffdc35a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":46,"up_from":74,"up_thru":216,"down_at":48,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6809","nonce":4293004354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6811","nonce":4293004354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6815","nonce":4293004354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":4293004354},{"type":"v1","addr":"172.21.15.63:6813","nonce":4293004354}]},"public_addr":"172.21.15.63:6809/4293004354","cluster_addr":"172.21.15.63:6811/4293004354","heartbeat_back_addr":"172.21.15.63:6815/4293004354","heartbeat_front_addr":"172.21.15.63:6813/4293004354","state":["exists","up"]},{"osd":4,"uuid":"1cec9702-fa09-4d7c-b111-7d10b1950cfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":142,"up_from":173,"up_thru":216,"down_at":144,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6801","nonce":4121301040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6803","nonce":4121301040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6807","nonce":4121301040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":4121301040},{"type":"v1","addr":"172.21.15.99:6805","nonce":4121301040}]},"public_addr":"172.21.15.99:6801/4121301040","cluster_addr":"172.21.15.99:6803/4121301040","heartbeat_back_addr":"172.21.15.99:6807/4121301040","heartbeat_front_addr":"172.21.15.99:6805/4121301040","state":["exists","up"]},{"osd":5,"uuid":"7c0b908a-65c9-426c-867c-cc8ed65084b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":179,"up_from":216,"up_thru":217,"down_at":181,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6809","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6811","nonce":1249382049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6813","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6815","nonce":1249382049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6821","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6823","nonce":1249382049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1249382049},{"type":"v1","addr":"172.21.15.99:6819","nonce":1249382049}]},"public_addr":"172.21.15.99:6811/1249382049","cluster_addr":"172.21.15.99:6815/1249382049","heartbeat_back_addr":"172.21.15.99:6823/1249382049","heartbeat_front_addr":"172.21.15.99:6819/1249382049","state":["exists","up"]},{"osd":6,"uuid":"78d5990a-4535-4110-ba04-88f01ab9fe28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6810","nonce":4152410871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6814","nonce":4152410871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6822","nonce":4152410871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":4152410871},{"type":"v1","addr":"172.21.15.99:6818","nonce":4152410871}]},"public_addr":"172.21.15.99:6810/4152410871","cluster_addr":"172.21.15.99:6814/4152410871","heartbeat_back_addr":"172.21.15.99:6822/4152410871","heartbeat_front_addr":"172.21.15.99:6818/4152410871","state":["exists","up"]},{"osd":7,"uuid":"53f8511c-729f-4423-8eac-813c01f8f72b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":216,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6825","nonce":2923799929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6827","nonce":2923799929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6831","nonce":2923799929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2923799929},{"type":"v1","addr":"172.21.15.99:6829","nonce":2923799929}]},"public_addr":"172.21.15.99:6825/2923799929","cluster_addr":"172.21.15.99:6827/2923799929","heartbeat_back_addr":"172.21.15.99:6831/2923799929","heartbeat_front_addr":"172.21.15.99:6829/2923799929","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.267895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.161313+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.447964+0000","dead_epoch":196},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:38.021320+0000","dead_epoch":48},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.883961+0000","dead_epoch":144},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.938146+0000","dead_epoch":181},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.627237+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:25:37.887523+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.18","osds":[2,5]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:30:14.179 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T09:30:14.179 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:14.350 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:14.350 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:14.350 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:14.364 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T09:30:14.689 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T09:30:14.700 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T09:30:14.700 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:14.848 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:14.848 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:14.848 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:14.862 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T09:30:15.175 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T09:30:15.190 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T09:30:15.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:15.343 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:15.343 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:15.343 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:15.357 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T09:30:15.671 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T09:30:15.683 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T09:30:15.683 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:15.836 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:15.836 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:15.836 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:15.849 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T09:30:16.163 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T09:30:16.177 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T09:30:16.177 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:16.325 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:16.325 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:16.325 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:16.339 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T09:30:16.661 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T09:30:16.673 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T09:30:16.673 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:16.831 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:16.831 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-13T09:30:16.831 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:16.845 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T09:30:17.153 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T09:30:17.165 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T09:30:17.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:17.317 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:17.317 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:17.317 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:17.331 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T09:30:17.637 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T09:30:17.648 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T09:30:17.649 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:30:17.803 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-09-13T09:30:17.803 INFO:teuthology.orchestra.run.smithi063.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:30:17.804 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-09-13T09:30:17.816 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T09:30:18.136 INFO:teuthology.orchestra.run.smithi063.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T09:30:18.146 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:18.455 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:18.458 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:18.471 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-13T09:30:16.596605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25463,"ondisk_log_size":25463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395192,"kb_used_data":22004,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579528,"statfs":{"total":773094113280,"available":772689436672,"internally_reserved":0,"allocated":22532096,"data_stored":12041654,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.163089"},"pg_stats":[{"pgid":"3.1f","version":"217'535","reported_seq":2244,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700567+0000","last_change":"2024-09-13T09:29:51.329628+0000","last_active":"2024-09-13T09:30:04.700567+0000","last_peered":"2024-09-13T09:30:04.700567+0000","last_clean":"2024-09-13T09:30:04.700567+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:04.700567+0000","last_undegraded":"2024-09-13T09:30:04.700567+0000","last_fullsized":"2024-09-13T09:30:04.700567+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:43.571566+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"209'831","reported_seq":5831,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433026+0000","last_change":"2024-09-13T09:29:44.372628+0000","last_active":"2024-09-13T09:30:05.433026+0000","last_peered":"2024-09-13T09:30:05.433026+0000","last_clean":"2024-09-13T09:30:05.433026+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:05.433026+0000","last_undegraded":"2024-09-13T09:30:05.433026+0000","last_fullsized":"2024-09-13T09:30:05.433026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":831,"log_dups_size":0,"ondisk_log_size":831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.939246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"209'629","reported_seq":5438,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748750+0000","last_change":"2024-09-13T09:29:52.302657+0000","last_active":"2024-09-13T09:30:06.748750+0000","last_peered":"2024-09-13T09:30:06.748750+0000","last_clean":"2024-09-13T09:30:06.748750+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:06.748750+0000","last_undegraded":"2024-09-13T09:30:06.748750+0000","last_fullsized":"2024-09-13T09:30:06.748750+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:14:50.114832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"212'819","reported_seq":6150,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748562+0000","last_change":"2024-09-13T09:29:52.330059+0000","last_active":"2024-09-13T09:30:06.748562+0000","last_peered":"2024-09-13T09:30:06.748562+0000","last_clean":"2024-09-13T09:30:06.748562+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:06.748562+0000","last_undegraded":"2024-09-13T09:30:06.748562+0000","last_fullsized":"2024-09-13T09:30:06.748562+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:01:38.645385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7169,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700079+0000","last_change":"2024-09-13T09:29:44.278432+0000","last_active":"2024-09-13T09:30:04.700079+0000","last_peered":"2024-09-13T09:30:04.700079+0000","last_clean":"2024-09-13T09:30:04.700079+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:04.700079+0000","last_undegraded":"2024-09-13T09:30:04.700079+0000","last_fullsized":"2024-09-13T09:30:04.700079+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:24.917547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"211'680","reported_seq":13797,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434186+0000","last_change":"2024-09-13T09:29:44.237452+0000","last_active":"2024-09-13T09:30:05.434186+0000","last_peered":"2024-09-13T09:30:05.434186+0000","last_clean":"2024-09-13T09:30:05.434186+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:05.434186+0000","last_undegraded":"2024-09-13T09:30:05.434186+0000","last_fullsized":"2024-09-13T09:30:05.434186+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:22.783828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"211'666","reported_seq":3868,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246820+0000","last_change":"2024-09-13T09:29:44.322458+0000","last_active":"2024-09-13T09:30:06.246820+0000","last_peered":"2024-09-13T09:30:06.246820+0000","last_clean":"2024-09-13T09:30:06.246820+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:06.246820+0000","last_undegraded":"2024-09-13T09:30:06.246820+0000","last_fullsized":"2024-09-13T09:30:06.246820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:31:04.526928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"212'691","reported_seq":5866,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.746774+0000","last_change":"2024-09-13T09:29:51.341364+0000","last_active":"2024-09-13T09:30:06.746774+0000","last_peered":"2024-09-13T09:30:06.746774+0000","last_clean":"2024-09-13T09:30:06.746774+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:06.746774+0000","last_undegraded":"2024-09-13T09:30:06.746774+0000","last_fullsized":"2024-09-13T09:30:06.746774+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:17:42.266961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"211'788","reported_seq":3015,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700432+0000","last_change":"2024-09-13T09:29:54.325150+0000","last_active":"2024-09-13T09:30:04.700432+0000","last_peered":"2024-09-13T09:30:04.700432+0000","last_clean":"2024-09-13T09:30:04.700432+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:04.700432+0000","last_undegraded":"2024-09-13T09:30:04.700432+0000","last_fullsized":"2024-09-13T09:30:04.700432+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:25.709713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6166,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748498+0000","last_change":"2024-09-13T09:29:51.321889+0000","last_active":"2024-09-13T09:30:06.748498+0000","last_peered":"2024-09-13T09:30:06.748498+0000","last_clean":"2024-09-13T09:30:06.748498+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:06.748498+0000","last_undegraded":"2024-09-13T09:30:06.748498+0000","last_fullsized":"2024-09-13T09:30:06.748498+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:12.873717+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"210'497","reported_seq":10587,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747787+0000","last_change":"2024-09-13T09:29:44.233280+0000","last_active":"2024-09-13T09:30:05.747787+0000","last_peered":"2024-09-13T09:30:05.747787+0000","last_clean":"2024-09-13T09:30:05.747787+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:05.747787+0000","last_undegraded":"2024-09-13T09:30:05.747787+0000","last_fullsized":"2024-09-13T09:30:05.747787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:13:17.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"210'613","reported_seq":6519,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434291+0000","last_change":"2024-09-13T09:29:44.238073+0000","last_active":"2024-09-13T09:30:05.434291+0000","last_peered":"2024-09-13T09:30:05.434291+0000","last_clean":"2024-09-13T09:30:05.434291+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:05.434291+0000","last_undegraded":"2024-09-13T09:30:05.434291+0000","last_fullsized":"2024-09-13T09:30:05.434291+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:34:02.710053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":240,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748370+0000","last_change":"2024-09-13T09:27:02.704369+0000","last_active":"2024-09-13T09:30:06.748370+0000","last_peered":"2024-09-13T09:30:06.748370+0000","last_clean":"2024-09-13T09:30:06.748370+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:06.748370+0000","last_undegraded":"2024-09-13T09:30:06.748370+0000","last_fullsized":"2024-09-13T09:30:06.748370+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:57.182597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"217'540","reported_seq":2909,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748496+0000","last_change":"2024-09-13T09:29:51.341689+0000","last_active":"2024-09-13T09:30:06.748496+0000","last_peered":"2024-09-13T09:30:06.748496+0000","last_clean":"2024-09-13T09:30:06.748496+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:06.748496+0000","last_undegraded":"2024-09-13T09:30:06.748496+0000","last_fullsized":"2024-09-13T09:30:06.748496+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:56.843638+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":248,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.749025+0000","last_change":"2024-09-13T09:26:39.540277+0000","last_active":"2024-09-13T09:30:06.749025+0000","last_peered":"2024-09-13T09:30:06.749025+0000","last_clean":"2024-09-13T09:30:06.749025+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:06.749025+0000","last_undegraded":"2024-09-13T09:30:06.749025+0000","last_fullsized":"2024-09-13T09:30:06.749025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:07:42.940640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"210'965","reported_seq":4809,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.610219+0000","last_change":"2024-09-13T09:29:44.322446+0000","last_active":"2024-09-13T09:30:06.610219+0000","last_peered":"2024-09-13T09:30:06.610219+0000","last_clean":"2024-09-13T09:30:06.610219+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:06.610219+0000","last_undegraded":"2024-09-13T09:30:06.610219+0000","last_fullsized":"2024-09-13T09:30:06.610219+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":965,"log_dups_size":0,"ondisk_log_size":965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:48:22.045121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434427+0000","last_change":"2024-09-13T09:29:40.201770+0000","last_active":"2024-09-13T09:30:05.434427+0000","last_peered":"2024-09-13T09:30:05.434427+0000","last_clean":"2024-09-13T09:30:05.434427+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:05.434427+0000","last_undegraded":"2024-09-13T09:30:05.434427+0000","last_fullsized":"2024-09-13T09:30:05.434427+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:07.863545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9190,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433166+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:05.433166+0000","last_peered":"2024-09-13T09:30:05.433166+0000","last_clean":"2024-09-13T09:30:05.433166+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:05.433166+0000","last_undegraded":"2024-09-13T09:30:05.433166+0000","last_fullsized":"2024-09-13T09:30:05.433166+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:45.458148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":200,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433208+0000","last_change":"2024-09-13T09:29:51.281204+0000","last_active":"2024-09-13T09:30:05.433208+0000","last_peered":"2024-09-13T09:30:05.433208+0000","last_clean":"2024-09-13T09:30:05.433208+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:05.433208+0000","last_undegraded":"2024-09-13T09:30:05.433208+0000","last_fullsized":"2024-09-13T09:30:05.433208+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:33.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"212'928","reported_seq":6275,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.700427+0000","last_change":"2024-09-13T09:29:44.278448+0000","last_active":"2024-09-13T09:30:04.700427+0000","last_peered":"2024-09-13T09:30:04.700427+0000","last_clean":"2024-09-13T09:30:04.700427+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:04.700427+0000","last_undegraded":"2024-09-13T09:30:04.700427+0000","last_fullsized":"2024-09-13T09:30:04.700427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:47.502979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"209'1424","reported_seq":4477,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699603+0000","last_change":"2024-09-13T09:29:44.275636+0000","last_active":"2024-09-13T09:30:04.699603+0000","last_peered":"2024-09-13T09:30:04.699603+0000","last_clean":"2024-09-13T09:30:04.699603+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:04.699603+0000","last_undegraded":"2024-09-13T09:30:04.699603+0000","last_fullsized":"2024-09-13T09:30:04.699603+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:27.580162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"19'117","reported_seq":371,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747540+0000","last_change":"2024-09-13T09:25:43.999556+0000","last_active":"2024-09-13T09:30:05.747540+0000","last_peered":"2024-09-13T09:30:05.747540+0000","last_clean":"2024-09-13T09:30:05.747540+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:05.747540+0000","last_undegraded":"2024-09-13T09:30:05.747540+0000","last_fullsized":"2024-09-13T09:30:05.747540+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:42.926610+0000","last_clean_scrub_stamp":"2024-09-13T09:25:42.926610+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:05:56.726413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"17'1","reported_seq":229,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246866+0000","last_change":"2024-09-13T09:25:44.975794+0000","last_active":"2024-09-13T09:30:06.246866+0000","last_peered":"2024-09-13T09:30:06.246866+0000","last_clean":"2024-09-13T09:30:06.246866+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:06.246866+0000","last_undegraded":"2024-09-13T09:30:06.246866+0000","last_fullsized":"2024-09-13T09:30:06.246866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:25:42.595421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9538,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.246633+0000","last_change":"2024-09-13T09:29:44.321698+0000","last_active":"2024-09-13T09:30:06.246633+0000","last_peered":"2024-09-13T09:30:06.246633+0000","last_clean":"2024-09-13T09:30:06.246633+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:06.246633+0000","last_undegraded":"2024-09-13T09:30:06.246633+0000","last_fullsized":"2024-09-13T09:30:06.246633+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:11.466841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"19'2","reported_seq":253,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747653+0000","last_change":"2024-09-13T09:26:39.544488+0000","last_active":"2024-09-13T09:30:05.747653+0000","last_peered":"2024-09-13T09:30:05.747653+0000","last_clean":"2024-09-13T09:30:05.747653+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:05.747653+0000","last_undegraded":"2024-09-13T09:30:05.747653+0000","last_fullsized":"2024-09-13T09:30:05.747653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:57.919179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"212'840","reported_seq":11367,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434421+0000","last_change":"2024-09-13T09:29:44.238109+0000","last_active":"2024-09-13T09:30:05.434421+0000","last_peered":"2024-09-13T09:30:05.434421+0000","last_clean":"2024-09-13T09:30:05.434421+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:05.434421+0000","last_undegraded":"2024-09-13T09:30:05.434421+0000","last_fullsized":"2024-09-13T09:30:05.434421+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:29:32.423203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":182,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748225+0000","last_change":"2024-09-13T09:29:51.279392+0000","last_active":"2024-09-13T09:30:06.748225+0000","last_peered":"2024-09-13T09:30:06.748225+0000","last_clean":"2024-09-13T09:30:06.748225+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:06.748225+0000","last_undegraded":"2024-09-13T09:30:06.748225+0000","last_fullsized":"2024-09-13T09:30:06.748225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:37.997623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7101,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432919+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:05.432919+0000","last_peered":"2024-09-13T09:30:05.432919+0000","last_clean":"2024-09-13T09:30:05.432919+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:05.432919+0000","last_undegraded":"2024-09-13T09:30:05.432919+0000","last_fullsized":"2024-09-13T09:30:05.432919+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:28:54.042990+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":243,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747532+0000","last_change":"2024-09-13T09:26:39.540521+0000","last_active":"2024-09-13T09:30:05.747532+0000","last_peered":"2024-09-13T09:30:05.747532+0000","last_clean":"2024-09-13T09:30:05.747532+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:05.747532+0000","last_undegraded":"2024-09-13T09:30:05.747532+0000","last_fullsized":"2024-09-13T09:30:05.747532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:41.923534+0000","last_clean_scrub_stamp":"2024-09-13T09:25:41.923534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:01.214653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"211'653","reported_seq":3505,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.432879+0000","last_change":"2024-09-13T09:29:44.370982+0000","last_active":"2024-09-13T09:30:05.432879+0000","last_peered":"2024-09-13T09:30:05.432879+0000","last_clean":"2024-09-13T09:30:05.432879+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:05.432879+0000","last_undegraded":"2024-09-13T09:30:05.432879+0000","last_fullsized":"2024-09-13T09:30:05.432879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:15:44.995461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"209'483","reported_seq":12049,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.434041+0000","last_change":"2024-09-13T09:29:44.237484+0000","last_active":"2024-09-13T09:30:05.434041+0000","last_peered":"2024-09-13T09:30:05.434041+0000","last_clean":"2024-09-13T09:30:05.434041+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:05.434041+0000","last_undegraded":"2024-09-13T09:30:05.434041+0000","last_fullsized":"2024-09-13T09:30:05.434041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:02:37.361009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"211'572","reported_seq":7649,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748310+0000","last_change":"2024-09-13T09:29:44.229060+0000","last_active":"2024-09-13T09:30:06.748310+0000","last_peered":"2024-09-13T09:30:06.748310+0000","last_clean":"2024-09-13T09:30:06.748310+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:06.748310+0000","last_undegraded":"2024-09-13T09:30:06.748310+0000","last_fullsized":"2024-09-13T09:30:06.748310+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:07.673817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"210'622","reported_seq":4506,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.433036+0000","last_change":"2024-09-13T09:29:44.371528+0000","last_active":"2024-09-13T09:30:05.433036+0000","last_peered":"2024-09-13T09:30:05.433036+0000","last_clean":"2024-09-13T09:30:05.433036+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:05.433036+0000","last_undegraded":"2024-09-13T09:30:05.433036+0000","last_fullsized":"2024-09-13T09:30:05.433036+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:25:49.579729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7640,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748768+0000","last_change":"2024-09-13T09:29:51.357978+0000","last_active":"2024-09-13T09:30:06.748768+0000","last_peered":"2024-09-13T09:30:06.748768+0000","last_clean":"2024-09-13T09:30:06.748768+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:06.748768+0000","last_undegraded":"2024-09-13T09:30:06.748768+0000","last_fullsized":"2024-09-13T09:30:06.748768+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:06:32.572241+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"217'498","reported_seq":14666,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747792+0000","last_change":"2024-09-13T09:29:51.366598+0000","last_active":"2024-09-13T09:30:05.747792+0000","last_peered":"2024-09-13T09:30:05.747792+0000","last_clean":"2024-09-13T09:30:05.747792+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:05.747792+0000","last_undegraded":"2024-09-13T09:30:05.747792+0000","last_fullsized":"2024-09-13T09:30:05.747792+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:25:28.046650+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"211'1205","reported_seq":9159,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747948+0000","last_change":"2024-09-13T09:29:44.236105+0000","last_active":"2024-09-13T09:30:05.747948+0000","last_peered":"2024-09-13T09:30:05.747948+0000","last_clean":"2024-09-13T09:30:05.747948+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:05.747948+0000","last_undegraded":"2024-09-13T09:30:05.747948+0000","last_fullsized":"2024-09-13T09:30:05.747948+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1205,"log_dups_size":0,"ondisk_log_size":1205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:30.493572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"211'483","reported_seq":7265,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747872+0000","last_change":"2024-09-13T09:29:44.236466+0000","last_active":"2024-09-13T09:30:05.747872+0000","last_peered":"2024-09-13T09:30:05.747872+0000","last_clean":"2024-09-13T09:30:05.747872+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:05.747872+0000","last_undegraded":"2024-09-13T09:30:05.747872+0000","last_fullsized":"2024-09-13T09:30:05.747872+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:15:24.511124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"211'887","reported_seq":4070,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.748769+0000","last_change":"2024-09-13T09:29:44.229062+0000","last_active":"2024-09-13T09:30:06.748769+0000","last_peered":"2024-09-13T09:30:06.748769+0000","last_clean":"2024-09-13T09:30:06.748769+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:06.748769+0000","last_undegraded":"2024-09-13T09:30:06.748769+0000","last_fullsized":"2024-09-13T09:30:06.748769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:44:29.356689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"212'895","reported_seq":9205,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747542+0000","last_change":"2024-09-13T09:29:44.235484+0000","last_active":"2024-09-13T09:30:05.747542+0000","last_peered":"2024-09-13T09:30:05.747542+0000","last_clean":"2024-09-13T09:30:05.747542+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:05.747542+0000","last_undegraded":"2024-09-13T09:30:05.747542+0000","last_fullsized":"2024-09-13T09:30:05.747542+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:42.588638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6241,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-13T09:30:04.699859+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:04.699859+0000","last_peered":"2024-09-13T09:30:04.699859+0000","last_clean":"2024-09-13T09:30:04.699859+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:04.699859+0000","last_undegraded":"2024-09-13T09:30:04.699859+0000","last_fullsized":"2024-09-13T09:30:04.699859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:11:43.688749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"209'1122","reported_seq":9794,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-13T09:30:05.747717+0000","last_change":"2024-09-13T09:29:44.234845+0000","last_active":"2024-09-13T09:30:05.747717+0000","last_peered":"2024-09-13T09:30:05.747717+0000","last_clean":"2024-09-13T09:30:05.747717+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:05.747717+0000","last_undegraded":"2024-09-13T09:30:05.747717+0000","last_fullsized":"2024-09-13T09:30:05.747717+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:30.144378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25343,"ondisk_log_size":25343,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51976,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319864,"statfs":{"total":96636764160,"available":96583540736,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.0369999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.626},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.86199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.89900000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.079}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.123}]}]},{"osd":6,"up_from":15,"seq":64424509498,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56000,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315840,"statfs":{"total":96636764160,"available":96579420160,"internally_reserved":0,"allocated":2801664,"data_stored":1358391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.874}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":1.127}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.048},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.99399999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.0760000000000001}]}]},{"osd":5,"up_from":216,"seq":927712935943,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]}]},{"osd":4,"up_from":173,"seq":743029342227,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.147},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.0629999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0069999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.97899999999999998}]}]},{"osd":3,"up_from":74,"seq":317827579945,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79924,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291916,"statfs":{"total":96636764160,"available":96554921984,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.46100000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.038},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.752},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.486}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.579},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.72699999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509498,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64260,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307580,"statfs":{"total":96636764160,"available":96570961920,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.432}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.57399999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.68899999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47804,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324036,"statfs":{"total":96636764160,"available":96587812864,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.502},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.753},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.0029999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.026},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":0.77900000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":0.878}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":1.0649999999999999}]}]},{"osd":2,"up_from":207,"seq":889058230281,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31292,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:18.476 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.476 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.476 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.477 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T09:25:42.926610+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=42, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.478 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T09:25:41.923534+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=41, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.479 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:18.480 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:30:38.482 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:38.807 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:38.809 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:38.822 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-13T09:30:38.603584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25512,"ondisk_log_size":25512,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395124,"kb_used_data":22004,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579596,"statfs":{"total":773094113280,"available":772689506304,"internally_reserved":0,"allocated":22532096,"data_stored":12041654,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003804"},"pg_stats":[{"pgid":"3.1f","version":"232'537","reported_seq":2258,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.814109+0000","last_change":"2024-09-13T09:30:20.814225+0000","last_active":"2024-09-13T09:30:20.814109+0000","last_peered":"2024-09-13T09:30:20.814109+0000","last_clean":"2024-09-13T09:30:20.814109+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:20.814109+0000","last_undegraded":"2024-09-13T09:30:20.814109+0000","last_fullsized":"2024-09-13T09:30:20.814109+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'537","last_scrub_stamp":"2024-09-13T09:30:20.814082+0000","last_deep_scrub":"232'537","last_deep_scrub_stamp":"2024-09-13T09:30:20.814082+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.814082+0000","objects_scrubbed":2,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:42.421891+0000","scrub_duration":0.0088729570000000008,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"232'832","reported_seq":5844,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:24.887597+0000","last_change":"2024-09-13T09:30:24.887824+0000","last_active":"2024-09-13T09:30:24.887597+0000","last_peered":"2024-09-13T09:30:24.887597+0000","last_clean":"2024-09-13T09:30:24.887597+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:24.887597+0000","last_undegraded":"2024-09-13T09:30:24.887597+0000","last_fullsized":"2024-09-13T09:30:24.887597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'832","last_scrub_stamp":"2024-09-13T09:30:24.887544+0000","last_deep_scrub":"232'832","last_deep_scrub_stamp":"2024-09-13T09:30:24.887544+0000","last_clean_scrub_stamp":"2024-09-13T09:30:24.887544+0000","objects_scrubbed":1,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:23.212633+0000","scrub_duration":0.0077204079999999998,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"232'630","reported_seq":5451,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:31.012490+0000","last_change":"2024-09-13T09:30:31.012793+0000","last_active":"2024-09-13T09:30:31.012490+0000","last_peered":"2024-09-13T09:30:31.012490+0000","last_clean":"2024-09-13T09:30:31.012490+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:31.012490+0000","last_undegraded":"2024-09-13T09:30:31.012490+0000","last_fullsized":"2024-09-13T09:30:31.012490+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'630","last_scrub_stamp":"2024-09-13T09:30:31.012442+0000","last_deep_scrub":"232'630","last_deep_scrub_stamp":"2024-09-13T09:30:31.012442+0000","last_clean_scrub_stamp":"2024-09-13T09:30:31.012442+0000","objects_scrubbed":1,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:11.524664+0000","scrub_duration":0.0067878080000000002,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"232'822","reported_seq":6165,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:32.964425+0000","last_change":"2024-09-13T09:30:32.964585+0000","last_active":"2024-09-13T09:30:32.964425+0000","last_peered":"2024-09-13T09:30:32.964425+0000","last_clean":"2024-09-13T09:30:32.964425+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:32.964425+0000","last_undegraded":"2024-09-13T09:30:32.964425+0000","last_fullsized":"2024-09-13T09:30:32.964425+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'822","last_scrub_stamp":"2024-09-13T09:30:32.964394+0000","last_deep_scrub":"232'822","last_deep_scrub_stamp":"2024-09-13T09:30:32.964394+0000","last_clean_scrub_stamp":"2024-09-13T09:30:32.964394+0000","objects_scrubbed":3,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:02:54.222598+0000","scrub_duration":0.010215768,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7181,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:27.937872+0000","last_change":"2024-09-13T09:30:27.938153+0000","last_active":"2024-09-13T09:30:27.937872+0000","last_peered":"2024-09-13T09:30:27.937872+0000","last_clean":"2024-09-13T09:30:27.937872+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:27.937872+0000","last_undegraded":"2024-09-13T09:30:27.937872+0000","last_fullsized":"2024-09-13T09:30:27.937872+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'680","last_scrub_stamp":"2024-09-13T09:30:27.937816+0000","last_deep_scrub":"192'680","last_deep_scrub_stamp":"2024-09-13T09:30:27.937816+0000","last_clean_scrub_stamp":"2024-09-13T09:30:27.937816+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:30.858036+0000","scrub_duration":0.0038834350000000002,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"232'681","reported_seq":13810,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:17.017006+0000","last_change":"2024-09-13T09:30:17.017298+0000","last_active":"2024-09-13T09:30:17.017006+0000","last_peered":"2024-09-13T09:30:17.017006+0000","last_clean":"2024-09-13T09:30:17.017006+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:17.017006+0000","last_undegraded":"2024-09-13T09:30:17.017006+0000","last_fullsized":"2024-09-13T09:30:17.017006+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'681","last_scrub_stamp":"2024-09-13T09:30:17.016955+0000","last_deep_scrub":"232'681","last_deep_scrub_stamp":"2024-09-13T09:30:17.016955+0000","last_clean_scrub_stamp":"2024-09-13T09:30:17.016955+0000","objects_scrubbed":1,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:12.805377+0000","scrub_duration":0.0063088859999999997,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"232'668","reported_seq":3882,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:23.864010+0000","last_change":"2024-09-13T09:30:23.864135+0000","last_active":"2024-09-13T09:30:23.864010+0000","last_peered":"2024-09-13T09:30:23.864010+0000","last_clean":"2024-09-13T09:30:23.864010+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:23.864010+0000","last_undegraded":"2024-09-13T09:30:23.864010+0000","last_fullsized":"2024-09-13T09:30:23.864010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'668","last_scrub_stamp":"2024-09-13T09:30:23.863982+0000","last_deep_scrub":"232'668","last_deep_scrub_stamp":"2024-09-13T09:30:23.863982+0000","last_clean_scrub_stamp":"2024-09-13T09:30:23.863982+0000","objects_scrubbed":2,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:22.547564+0000","scrub_duration":0.0086823170000000002,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"232'694","reported_seq":5881,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.049996+0000","last_change":"2024-09-13T09:30:20.050220+0000","last_active":"2024-09-13T09:30:20.049996+0000","last_peered":"2024-09-13T09:30:20.049996+0000","last_clean":"2024-09-13T09:30:20.049996+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:20.049996+0000","last_undegraded":"2024-09-13T09:30:20.049996+0000","last_fullsized":"2024-09-13T09:30:20.049996+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'694","last_scrub_stamp":"2024-09-13T09:30:20.049943+0000","last_deep_scrub":"232'694","last_deep_scrub_stamp":"2024-09-13T09:30:20.049943+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.049943+0000","objects_scrubbed":3,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:30.525532+0000","scrub_duration":0.0095315230000000001,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"232'790","reported_seq":3029,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:22.836956+0000","last_change":"2024-09-13T09:30:22.837331+0000","last_active":"2024-09-13T09:30:22.836956+0000","last_peered":"2024-09-13T09:30:22.836956+0000","last_clean":"2024-09-13T09:30:22.836956+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:22.836956+0000","last_undegraded":"2024-09-13T09:30:22.836956+0000","last_fullsized":"2024-09-13T09:30:22.836956+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'790","last_scrub_stamp":"2024-09-13T09:30:22.836841+0000","last_deep_scrub":"232'790","last_deep_scrub_stamp":"2024-09-13T09:30:22.836841+0000","last_clean_scrub_stamp":"2024-09-13T09:30:22.836841+0000","objects_scrubbed":2,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:29.200769+0000","scrub_duration":0.0085375950000000003,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6178,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:25.026845+0000","last_change":"2024-09-13T09:30:25.027102+0000","last_active":"2024-09-13T09:30:25.026845+0000","last_peered":"2024-09-13T09:30:25.026845+0000","last_clean":"2024-09-13T09:30:25.026845+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:25.026845+0000","last_undegraded":"2024-09-13T09:30:25.026845+0000","last_fullsized":"2024-09-13T09:30:25.026845+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'605","last_scrub_stamp":"2024-09-13T09:30:25.026796+0000","last_deep_scrub":"217'605","last_deep_scrub_stamp":"2024-09-13T09:30:25.026796+0000","last_clean_scrub_stamp":"2024-09-13T09:30:25.026796+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:20.024564+0000","scrub_duration":0.003653281,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"232'498","reported_seq":10600,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:30.744365+0000","last_change":"2024-09-13T09:30:30.744599+0000","last_active":"2024-09-13T09:30:30.744365+0000","last_peered":"2024-09-13T09:30:30.744365+0000","last_clean":"2024-09-13T09:30:30.744365+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:30.744365+0000","last_undegraded":"2024-09-13T09:30:30.744365+0000","last_fullsized":"2024-09-13T09:30:30.744365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'498","last_scrub_stamp":"2024-09-13T09:30:30.744316+0000","last_deep_scrub":"232'498","last_deep_scrub_stamp":"2024-09-13T09:30:30.744316+0000","last_clean_scrub_stamp":"2024-09-13T09:30:30.744316+0000","objects_scrubbed":1,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:27.863929+0000","scrub_duration":0.0069570109999999999,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"232'614","reported_seq":6532,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:24.056515+0000","last_change":"2024-09-13T09:30:24.056787+0000","last_active":"2024-09-13T09:30:24.056515+0000","last_peered":"2024-09-13T09:30:24.056515+0000","last_clean":"2024-09-13T09:30:24.056515+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:24.056515+0000","last_undegraded":"2024-09-13T09:30:24.056515+0000","last_fullsized":"2024-09-13T09:30:24.056515+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'614","last_scrub_stamp":"2024-09-13T09:30:24.056465+0000","last_deep_scrub":"232'614","last_deep_scrub_stamp":"2024-09-13T09:30:24.056465+0000","last_clean_scrub_stamp":"2024-09-13T09:30:24.056465+0000","objects_scrubbed":1,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:25.566240+0000","scrub_duration":0.0075038919999999999,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":252,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:17.667463+0000","last_change":"2024-09-13T09:30:17.667797+0000","last_active":"2024-09-13T09:30:17.667463+0000","last_peered":"2024-09-13T09:30:17.667463+0000","last_clean":"2024-09-13T09:30:17.667463+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:17.667463+0000","last_undegraded":"2024-09-13T09:30:17.667463+0000","last_fullsized":"2024-09-13T09:30:17.667463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:17.667384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:17.667384+0000","last_clean_scrub_stamp":"2024-09-13T09:30:17.667384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:49.179677+0000","scrub_duration":0.0037910259999999999,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"232'541","reported_seq":2922,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:24.038595+0000","last_change":"2024-09-13T09:30:24.038843+0000","last_active":"2024-09-13T09:30:24.038595+0000","last_peered":"2024-09-13T09:30:24.038595+0000","last_clean":"2024-09-13T09:30:24.038595+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:24.038595+0000","last_undegraded":"2024-09-13T09:30:24.038595+0000","last_fullsized":"2024-09-13T09:30:24.038595+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'541","last_scrub_stamp":"2024-09-13T09:30:24.038543+0000","last_deep_scrub":"232'541","last_deep_scrub_stamp":"2024-09-13T09:30:24.038543+0000","last_clean_scrub_stamp":"2024-09-13T09:30:24.038543+0000","objects_scrubbed":1,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:58.883135+0000","scrub_duration":0.0075833929999999999,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":260,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.628611+0000","last_change":"2024-09-13T09:30:18.628981+0000","last_active":"2024-09-13T09:30:18.628611+0000","last_peered":"2024-09-13T09:30:18.628611+0000","last_clean":"2024-09-13T09:30:18.628611+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:18.628611+0000","last_undegraded":"2024-09-13T09:30:18.628611+0000","last_fullsized":"2024-09-13T09:30:18.628611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:18.628521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:18.628521+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.628521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:29:27.378445+0000","scrub_duration":0.0038613499999999999,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"232'966","reported_seq":4822,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.912256+0000","last_change":"2024-09-13T09:30:21.912588+0000","last_active":"2024-09-13T09:30:21.912256+0000","last_peered":"2024-09-13T09:30:21.912256+0000","last_clean":"2024-09-13T09:30:21.912256+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:21.912256+0000","last_undegraded":"2024-09-13T09:30:21.912256+0000","last_fullsized":"2024-09-13T09:30:21.912256+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'966","last_scrub_stamp":"2024-09-13T09:30:21.912207+0000","last_deep_scrub":"232'966","last_deep_scrub_stamp":"2024-09-13T09:30:21.912207+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.912207+0000","objects_scrubbed":1,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:34.233136+0000","scrub_duration":0.0077823739999999999,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.006475+0000","last_change":"2024-09-13T09:30:18.006732+0000","last_active":"2024-09-13T09:30:18.006475+0000","last_peered":"2024-09-13T09:30:18.006475+0000","last_clean":"2024-09-13T09:30:18.006475+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:18.006475+0000","last_undegraded":"2024-09-13T09:30:18.006475+0000","last_fullsized":"2024-09-13T09:30:18.006475+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:18.006431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:18.006431+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.006431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:42.425851+0000","scrub_duration":0.003406259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"209'880","reported_seq":9191,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.788076+0000","last_change":"2024-09-13T09:29:44.372371+0000","last_active":"2024-09-13T09:30:06.788076+0000","last_peered":"2024-09-13T09:30:06.788076+0000","last_clean":"2024-09-13T09:30:06.788076+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:06.788076+0000","last_undegraded":"2024-09-13T09:30:06.788076+0000","last_fullsized":"2024-09-13T09:30:06.788076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":212,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:16.859042+0000","last_change":"2024-09-13T09:30:16.859146+0000","last_active":"2024-09-13T09:30:16.859042+0000","last_peered":"2024-09-13T09:30:16.859042+0000","last_clean":"2024-09-13T09:30:16.859042+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:16.859042+0000","last_undegraded":"2024-09-13T09:30:16.859042+0000","last_fullsized":"2024-09-13T09:30:16.859042+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:16.859023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:16.859023+0000","last_clean_scrub_stamp":"2024-09-13T09:30:16.859023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:39.958454+0000","scrub_duration":0.002657745,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"232'930","reported_seq":6289,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:25.876119+0000","last_change":"2024-09-13T09:30:25.876395+0000","last_active":"2024-09-13T09:30:25.876119+0000","last_peered":"2024-09-13T09:30:25.876119+0000","last_clean":"2024-09-13T09:30:25.876119+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:25.876119+0000","last_undegraded":"2024-09-13T09:30:25.876119+0000","last_fullsized":"2024-09-13T09:30:25.876119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'930","last_scrub_stamp":"2024-09-13T09:30:25.876059+0000","last_deep_scrub":"232'930","last_deep_scrub_stamp":"2024-09-13T09:30:25.876059+0000","last_clean_scrub_stamp":"2024-09-13T09:30:25.876059+0000","objects_scrubbed":2,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:45.712490+0000","scrub_duration":0.010502780999999999,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"232'1427","reported_seq":4492,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:32.992426+0000","last_change":"2024-09-13T09:30:32.992829+0000","last_active":"2024-09-13T09:30:32.992426+0000","last_peered":"2024-09-13T09:30:32.992426+0000","last_clean":"2024-09-13T09:30:32.992426+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:32.992426+0000","last_undegraded":"2024-09-13T09:30:32.992426+0000","last_fullsized":"2024-09-13T09:30:32.992426+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'1427","last_scrub_stamp":"2024-09-13T09:30:32.992359+0000","last_deep_scrub":"232'1427","last_deep_scrub_stamp":"2024-09-13T09:30:32.992359+0000","last_clean_scrub_stamp":"2024-09-13T09:30:32.992359+0000","objects_scrubbed":3,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:38.667370+0000","scrub_duration":0.010444729,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"232'119","reported_seq":385,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.813403+0000","last_change":"2024-09-13T09:30:20.813706+0000","last_active":"2024-09-13T09:30:20.813403+0000","last_peered":"2024-09-13T09:30:20.813403+0000","last_clean":"2024-09-13T09:30:20.813403+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:20.813403+0000","last_undegraded":"2024-09-13T09:30:20.813403+0000","last_fullsized":"2024-09-13T09:30:20.813403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'119","last_scrub_stamp":"2024-09-13T09:30:20.813350+0000","last_deep_scrub":"232'119","last_deep_scrub_stamp":"2024-09-13T09:30:20.813350+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.813350+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:26:53.848530+0000","scrub_duration":0.010415627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"232'2","reported_seq":242,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.872171+0000","last_change":"2024-09-13T09:30:18.872412+0000","last_active":"2024-09-13T09:30:18.872171+0000","last_peered":"2024-09-13T09:30:18.872171+0000","last_clean":"2024-09-13T09:30:18.872171+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:18.872171+0000","last_undegraded":"2024-09-13T09:30:18.872171+0000","last_fullsized":"2024-09-13T09:30:18.872171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'2","last_scrub_stamp":"2024-09-13T09:30:18.872122+0000","last_deep_scrub":"232'2","last_deep_scrub_stamp":"2024-09-13T09:30:18.872122+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.872122+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:16.029753+0000","scrub_duration":0.006480754,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9550,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:26.880365+0000","last_change":"2024-09-13T09:30:26.880744+0000","last_active":"2024-09-13T09:30:26.880365+0000","last_peered":"2024-09-13T09:30:26.880365+0000","last_clean":"2024-09-13T09:30:26.880365+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:26.880365+0000","last_undegraded":"2024-09-13T09:30:26.880365+0000","last_fullsized":"2024-09-13T09:30:26.880365+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'1021","last_scrub_stamp":"2024-09-13T09:30:26.880257+0000","last_deep_scrub":"211'1021","last_deep_scrub_stamp":"2024-09-13T09:30:26.880257+0000","last_clean_scrub_stamp":"2024-09-13T09:30:26.880257+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:38:15.127399+0000","scrub_duration":0.0038710429999999998,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"232'3","reported_seq":266,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.809480+0000","last_change":"2024-09-13T09:30:18.809736+0000","last_active":"2024-09-13T09:30:18.809480+0000","last_peered":"2024-09-13T09:30:18.809480+0000","last_clean":"2024-09-13T09:30:18.809480+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:18.809480+0000","last_undegraded":"2024-09-13T09:30:18.809480+0000","last_fullsized":"2024-09-13T09:30:18.809480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'3","last_scrub_stamp":"2024-09-13T09:30:18.809433+0000","last_deep_scrub":"232'3","last_deep_scrub_stamp":"2024-09-13T09:30:18.809433+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.809433+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:22.407760+0000","scrub_duration":0.006879736,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"232'843","reported_seq":11382,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:27.126537+0000","last_change":"2024-09-13T09:30:27.126833+0000","last_active":"2024-09-13T09:30:27.126537+0000","last_peered":"2024-09-13T09:30:27.126537+0000","last_clean":"2024-09-13T09:30:27.126537+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:27.126537+0000","last_undegraded":"2024-09-13T09:30:27.126537+0000","last_fullsized":"2024-09-13T09:30:27.126537+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'843","last_scrub_stamp":"2024-09-13T09:30:27.126466+0000","last_deep_scrub":"232'843","last_deep_scrub_stamp":"2024-09-13T09:30:27.126466+0000","last_clean_scrub_stamp":"2024-09-13T09:30:27.126466+0000","objects_scrubbed":3,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:33.427533+0000","scrub_duration":0.010701713999999999,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":194,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.066262+0000","last_change":"2024-09-13T09:30:21.066461+0000","last_active":"2024-09-13T09:30:21.066262+0000","last_peered":"2024-09-13T09:30:21.066262+0000","last_clean":"2024-09-13T09:30:21.066262+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:21.066262+0000","last_undegraded":"2024-09-13T09:30:21.066262+0000","last_fullsized":"2024-09-13T09:30:21.066262+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:21.066217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:21.066217+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.066217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:43.942714+0000","scrub_duration":0.0035654110000000001,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"217'2104","reported_seq":7102,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.787793+0000","last_change":"2024-09-13T09:29:51.381492+0000","last_active":"2024-09-13T09:30:06.787793+0000","last_peered":"2024-09-13T09:30:06.787793+0000","last_clean":"2024-09-13T09:30:06.787793+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:06.787793+0000","last_undegraded":"2024-09-13T09:30:06.787793+0000","last_fullsized":"2024-09-13T09:30:06.787793+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":2104,"log_dups_size":0,"ondisk_log_size":2104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":255,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:19.843861+0000","last_change":"2024-09-13T09:30:19.844112+0000","last_active":"2024-09-13T09:30:19.843861+0000","last_peered":"2024-09-13T09:30:19.843861+0000","last_clean":"2024-09-13T09:30:19.843861+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:19.843861+0000","last_undegraded":"2024-09-13T09:30:19.843861+0000","last_fullsized":"2024-09-13T09:30:19.843861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:19.843817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:19.843817+0000","last_clean_scrub_stamp":"2024-09-13T09:30:19.843817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:47.416004+0000","scrub_duration":0.0035090439999999998,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"232'654","reported_seq":3518,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:22.850990+0000","last_change":"2024-09-13T09:30:22.851306+0000","last_active":"2024-09-13T09:30:22.850990+0000","last_peered":"2024-09-13T09:30:22.850990+0000","last_clean":"2024-09-13T09:30:22.850990+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:22.850990+0000","last_undegraded":"2024-09-13T09:30:22.850990+0000","last_fullsized":"2024-09-13T09:30:22.850990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'654","last_scrub_stamp":"2024-09-13T09:30:22.850943+0000","last_deep_scrub":"232'654","last_deep_scrub_stamp":"2024-09-13T09:30:22.850943+0000","last_clean_scrub_stamp":"2024-09-13T09:30:22.850943+0000","objects_scrubbed":1,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:20.209409+0000","scrub_duration":0.0071269209999999996,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"232'488","reported_seq":12066,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.072775+0000","last_change":"2024-09-13T09:30:21.073021+0000","last_active":"2024-09-13T09:30:21.072775+0000","last_peered":"2024-09-13T09:30:21.072775+0000","last_clean":"2024-09-13T09:30:21.072775+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:21.072775+0000","last_undegraded":"2024-09-13T09:30:21.072775+0000","last_fullsized":"2024-09-13T09:30:21.072775+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'488","last_scrub_stamp":"2024-09-13T09:30:21.072709+0000","last_deep_scrub":"232'488","last_deep_scrub_stamp":"2024-09-13T09:30:21.072709+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.072709+0000","objects_scrubbed":5,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:49.706218+0000","scrub_duration":0.011994694,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"232'574","reported_seq":7664,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.719028+0000","last_change":"2024-09-13T09:30:21.719342+0000","last_active":"2024-09-13T09:30:21.719028+0000","last_peered":"2024-09-13T09:30:21.719028+0000","last_clean":"2024-09-13T09:30:21.719028+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:21.719028+0000","last_undegraded":"2024-09-13T09:30:21.719028+0000","last_fullsized":"2024-09-13T09:30:21.719028+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'574","last_scrub_stamp":"2024-09-13T09:30:21.718979+0000","last_deep_scrub":"232'574","last_deep_scrub_stamp":"2024-09-13T09:30:21.718979+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.718979+0000","objects_scrubbed":2,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:07.720115+0000","scrub_duration":0.0080011579999999995,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"232'623","reported_seq":4519,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.869500+0000","last_change":"2024-09-13T09:30:20.869822+0000","last_active":"2024-09-13T09:30:20.869500+0000","last_peered":"2024-09-13T09:30:20.869500+0000","last_clean":"2024-09-13T09:30:20.869500+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:20.869500+0000","last_undegraded":"2024-09-13T09:30:20.869500+0000","last_fullsized":"2024-09-13T09:30:20.869500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'623","last_scrub_stamp":"2024-09-13T09:30:20.869450+0000","last_deep_scrub":"232'623","last_deep_scrub_stamp":"2024-09-13T09:30:20.869450+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.869450+0000","objects_scrubbed":1,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:29:30.985753+0000","scrub_duration":0.0067024739999999999,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7652,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:29.014610+0000","last_change":"2024-09-13T09:30:29.014861+0000","last_active":"2024-09-13T09:30:29.014610+0000","last_peered":"2024-09-13T09:30:29.014610+0000","last_clean":"2024-09-13T09:30:29.014610+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:29.014610+0000","last_undegraded":"2024-09-13T09:30:29.014610+0000","last_fullsized":"2024-09-13T09:30:29.014610+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'596","last_scrub_stamp":"2024-09-13T09:30:29.014557+0000","last_deep_scrub":"217'596","last_deep_scrub_stamp":"2024-09-13T09:30:29.014557+0000","last_clean_scrub_stamp":"2024-09-13T09:30:29.014557+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:13.992077+0000","scrub_duration":0.0038990359999999998,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"232'500","reported_seq":14681,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.776122+0000","last_change":"2024-09-13T09:30:21.776122+0000","last_active":"2024-09-13T09:30:21.776122+0000","last_peered":"2024-09-13T09:30:21.776122+0000","last_clean":"2024-09-13T09:30:21.776122+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:21.776122+0000","last_undegraded":"2024-09-13T09:30:21.776122+0000","last_fullsized":"2024-09-13T09:30:21.776122+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'500","last_scrub_stamp":"2024-09-13T09:30:21.775794+0000","last_deep_scrub":"232'500","last_deep_scrub_stamp":"2024-09-13T09:30:21.775794+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.775794+0000","objects_scrubbed":2,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:30.525491+0000","scrub_duration":0.007365595,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"232'1207","reported_seq":9174,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:27.719680+0000","last_change":"2024-09-13T09:30:27.719814+0000","last_active":"2024-09-13T09:30:27.719680+0000","last_peered":"2024-09-13T09:30:27.719680+0000","last_clean":"2024-09-13T09:30:27.719680+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:27.719680+0000","last_undegraded":"2024-09-13T09:30:27.719680+0000","last_fullsized":"2024-09-13T09:30:27.719680+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'1207","last_scrub_stamp":"2024-09-13T09:30:27.719652+0000","last_deep_scrub":"232'1207","last_deep_scrub_stamp":"2024-09-13T09:30:27.719652+0000","last_clean_scrub_stamp":"2024-09-13T09:30:27.719652+0000","objects_scrubbed":2,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:53.584551+0000","scrub_duration":0.0088464070000000006,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"232'484","reported_seq":7279,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:29.786664+0000","last_change":"2024-09-13T09:30:29.786879+0000","last_active":"2024-09-13T09:30:29.786664+0000","last_peered":"2024-09-13T09:30:29.786664+0000","last_clean":"2024-09-13T09:30:29.786664+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:29.786664+0000","last_undegraded":"2024-09-13T09:30:29.786664+0000","last_fullsized":"2024-09-13T09:30:29.786664+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'484","last_scrub_stamp":"2024-09-13T09:30:29.786616+0000","last_deep_scrub":"232'484","last_deep_scrub_stamp":"2024-09-13T09:30:29.786616+0000","last_clean_scrub_stamp":"2024-09-13T09:30:29.786616+0000","objects_scrubbed":1,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:10.267279+0000","scrub_duration":0.0066261050000000002,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"232'888","reported_seq":4084,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.680788+0000","last_change":"2024-09-13T09:30:20.680788+0000","last_active":"2024-09-13T09:30:20.680788+0000","last_peered":"2024-09-13T09:30:20.680788+0000","last_clean":"2024-09-13T09:30:20.680788+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:20.680788+0000","last_undegraded":"2024-09-13T09:30:20.680788+0000","last_fullsized":"2024-09-13T09:30:20.680788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'888","last_scrub_stamp":"2024-09-13T09:30:20.680505+0000","last_deep_scrub":"232'888","last_deep_scrub_stamp":"2024-09-13T09:30:20.680505+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.680505+0000","objects_scrubbed":1,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:13.433630+0000","scrub_duration":0.0089656230000000007,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"232'897","reported_seq":9220,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:22.774863+0000","last_change":"2024-09-13T09:30:22.774863+0000","last_active":"2024-09-13T09:30:22.774863+0000","last_peered":"2024-09-13T09:30:22.774863+0000","last_clean":"2024-09-13T09:30:22.774863+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:22.774863+0000","last_undegraded":"2024-09-13T09:30:22.774863+0000","last_fullsized":"2024-09-13T09:30:22.774863+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'897","last_scrub_stamp":"2024-09-13T09:30:22.774515+0000","last_deep_scrub":"232'897","last_deep_scrub_stamp":"2024-09-13T09:30:22.774515+0000","last_clean_scrub_stamp":"2024-09-13T09:30:22.774515+0000","objects_scrubbed":2,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:32.820000+0000","scrub_duration":0.0083601460000000006,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"209'591","reported_seq":6242,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:06.435735+0000","last_change":"2024-09-13T09:29:44.275577+0000","last_active":"2024-09-13T09:30:06.435735+0000","last_peered":"2024-09-13T09:30:06.435735+0000","last_clean":"2024-09-13T09:30:06.435735+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:06.435735+0000","last_undegraded":"2024-09-13T09:30:06.435735+0000","last_fullsized":"2024-09-13T09:30:06.435735+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:25:52.138270+0000","last_clean_scrub_stamp":"2024-09-13T09:25:52.138270+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"232'1123","reported_seq":9808,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:28.743148+0000","last_change":"2024-09-13T09:30:28.743148+0000","last_active":"2024-09-13T09:30:28.743148+0000","last_peered":"2024-09-13T09:30:28.743148+0000","last_clean":"2024-09-13T09:30:28.743148+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:28.743148+0000","last_undegraded":"2024-09-13T09:30:28.743148+0000","last_fullsized":"2024-09-13T09:30:28.743148+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'1123","last_scrub_stamp":"2024-09-13T09:30:28.742803+0000","last_deep_scrub":"232'1123","last_deep_scrub_stamp":"2024-09-13T09:30:28.742803+0000","last_clean_scrub_stamp":"2024-09-13T09:30:28.742803+0000","objects_scrubbed":1,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:37.618714+0000","scrub_duration":0.0072346880000000004,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25388,"ondisk_log_size":25388,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51964,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319876,"statfs":{"total":96636764160,"available":96583553024,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.77000000000000002},"min":{"1min":0.42799999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.86399999999999999,"5min":1.256,"15min":1.256},"last":1.143},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.49199999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94999999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.755,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.218},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.59499999999999997,"5min":0.375,"15min":0.375},"max":{"1min":1.1100000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83699999999999997,"15min":0.85499999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":1.042}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":1.2430000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.875}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.113},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.96299999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:28:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.92200000000000004,"15min":0.96699999999999997},"min":{"1min":0.438,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.276},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.96699999999999997,"15min":1.0169999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.282,"5min":1.357,"15min":1.357},"last":1.3300000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:29:45 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.96099999999999997,"15min":1.0389999999999999},"min":{"1min":0.59399999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0669999999999999,"5min":1.496,"15min":1.496},"last":1.087},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0049999999999999},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.151,"5min":1.573,"15min":1.573},"last":1.1819999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55992,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315848,"statfs":{"total":96636764160,"available":96579428352,"internally_reserved":0,"allocated":2801664,"data_stored":1358391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.79400000000000004,"15min":0.872},"min":{"1min":0.29899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0700000000000001,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76000000000000001,"15min":0.71899999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98899999999999999,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.86099999999999999,"15min":0.94299999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0469999999999999,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.79700000000000004,"15min":0.81699999999999995},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.85199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.911,"5min":2.911,"15min":2.911},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.97299999999999998,"15min":1.0209999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.254,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.032}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":0.877},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.97699999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.75},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.125}]},{"osd":5,"last update":"Fri Sep 13 09:28:47 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.46600000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.974,"5min":2.032,"15min":2.032},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0840000000000001,"15min":1.1140000000000001},"min":{"1min":0.496,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":3.343,"5min":5.0709999999999997,"15min":5.0709999999999997},"last":0.92100000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89900000000000002},"min":{"1min":0.48399999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0409999999999999,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":1.1180000000000001,"15min":1.3240000000000001},"min":{"1min":0.54800000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.153,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":1.0029999999999999}]}]},{"osd":5,"up_from":216,"seq":927712935947,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":4,"up_from":173,"seq":743029342232,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.1339999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.21}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":3,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.28},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.333},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.109}]},{"osd":7,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.425},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.514}]}]},{"osd":3,"up_from":74,"seq":317827579949,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79920,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291920,"statfs":{"total":96636764160,"available":96554926080,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.093},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.874}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.012},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:29:02 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.41299999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.46300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:29:51 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.151}]},{"osd":5,"last update":"Fri Sep 13 09:28:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.1830000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.91900000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.121},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":0.96499999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.26900000000000002,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.98699999999999999,"5min":1.484,"15min":1.484},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.36399999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65800000000000003},"min":{"1min":0.35799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.93500000000000005,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1930000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.25}]},{"osd":4,"last update":"Fri Sep 13 09:29:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.03},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.163}]},{"osd":5,"last update":"Fri Sep 13 09:28:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.97599999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.075,"5min":7.6879999999999997,"15min":7.6879999999999997},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.03},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.3320000000000001,"5min":1.9610000000000001,"15min":1.9610000000000001},"last":1.2969999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0029999999999999,"15min":1.077},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2689999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.96499999999999997,"15min":1.0249999999999999},"min":{"1min":0.47799999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.3180000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:29:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.044},"min":{"1min":0.56799999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.474,"5min":2.048,"15min":2.048},"last":1.6910000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.0589999999999999,"15min":1.137},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.379,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.819}]}]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47800,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324040,"statfs":{"total":96636764160,"available":96587816960,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72099999999999997,"15min":0.66900000000000004},"min":{"1min":0.33300000000000002,"5min":0.245,"15min":0.245},"max":{"1min":1.165,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66400000000000003,"15min":0.70799999999999996},"min":{"1min":0.36799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0940000000000001,"5min":1.333,"15min":1.333},"last":0.91300000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69099999999999995,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.99399999999999999,"5min":1.153,"15min":1.153},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.999,"5min":0.872,"15min":0.90800000000000003},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":7.5030000000000001,"5min":7.5030000000000001,"15min":7.5030000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.95599999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:29:49 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.016}]},{"osd":5,"last update":"Fri Sep 13 09:28:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.97699999999999998,"15min":1.0229999999999999},"min":{"1min":0.497,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1279999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.294,"5min":1.018,"15min":0.97999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":8.0109999999999992,"5min":8.0109999999999992,"15min":8.0109999999999992},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":1.0880000000000001,"15min":1.0940000000000001},"min":{"1min":0.53000000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.272,"5min":12.519,"15min":12.519},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.99299999999999999,"15min":1.0580000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.214,"5min":1.514,"15min":1.514},"last":1.052}]},{"osd":7,"last update":"Fri Sep 13 09:29:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1579999999999999,"15min":1.135},"min":{"1min":0.68899999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5589999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":1.024,"15min":1.101},"min":{"1min":0.59399999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.194,"5min":1.891,"15min":1.891},"last":0.88800000000000001}]}]},{"osd":2,"up_from":207,"seq":889058230285,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:38.827 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:38.827 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:38.827 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T09:25:52.138270+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=25, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=30, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:30:38.827 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:30:58.828 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:30:59.156 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-09-13T09:30:59.158 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-09-13T09:30:59.170 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-13T09:30:58.610174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":44,"num_read":251397,"num_read_kb":753907,"num_write":31690,"num_write_kb":441471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8888,"num_bytes_recovered":387711977,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25515,"ondisk_log_size":25515,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":395124,"kb_used_data":22004,"kb_used_omap":107,"kb_used_meta":372948,"kb_avail":754579596,"statfs":{"total":773094113280,"available":772689506304,"internally_reserved":0,"allocated":22532096,"data_stored":12041654,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":110234,"internal_metadata":381899110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003864"},"pg_stats":[{"pgid":"3.1f","version":"232'537","reported_seq":2259,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.814225+0000","last_change":"2024-09-13T09:30:20.814225+0000","last_active":"2024-09-13T09:30:20.814225+0000","last_peered":"2024-09-13T09:30:20.814225+0000","last_clean":"2024-09-13T09:30:20.814225+0000","last_became_active":"2024-09-13T09:29:51.281285+0000","last_became_peered":"2024-09-13T09:29:51.281285+0000","last_unstale":"2024-09-13T09:30:20.814225+0000","last_undegraded":"2024-09-13T09:30:20.814225+0000","last_fullsized":"2024-09-13T09:30:20.814225+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'537","last_scrub_stamp":"2024-09-13T09:30:20.814082+0000","last_deep_scrub":"232'537","last_deep_scrub_stamp":"2024-09-13T09:30:20.814082+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.814082+0000","objects_scrubbed":2,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:38:42.421891+0000","scrub_duration":0.0088729570000000008,"objects_trimmed":8,"snaptrim_duration":0.026327897999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3181,"num_read_kb":16243,"num_write":521,"num_write_kb":8046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":492659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"232'832","reported_seq":5844,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:24.887597+0000","last_change":"2024-09-13T09:30:24.887824+0000","last_active":"2024-09-13T09:30:24.887597+0000","last_peered":"2024-09-13T09:30:24.887597+0000","last_clean":"2024-09-13T09:30:24.887597+0000","last_became_active":"2024-09-13T09:27:05.741981+0000","last_became_peered":"2024-09-13T09:27:05.741981+0000","last_unstale":"2024-09-13T09:30:24.887597+0000","last_undegraded":"2024-09-13T09:30:24.887597+0000","last_fullsized":"2024-09-13T09:30:24.887597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'832","last_scrub_stamp":"2024-09-13T09:30:24.887544+0000","last_deep_scrub":"232'832","last_deep_scrub_stamp":"2024-09-13T09:30:24.887544+0000","last_clean_scrub_stamp":"2024-09-13T09:30:24.887544+0000","objects_scrubbed":1,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:23.212633+0000","scrub_duration":0.0077204079999999998,"objects_trimmed":0,"snaptrim_duration":0.0024877409999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6811,"num_read_kb":14464,"num_write":1130,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":1457245,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"232'630","reported_seq":5451,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:31.012490+0000","last_change":"2024-09-13T09:30:31.012793+0000","last_active":"2024-09-13T09:30:31.012490+0000","last_peered":"2024-09-13T09:30:31.012490+0000","last_clean":"2024-09-13T09:30:31.012490+0000","last_became_active":"2024-09-13T09:29:52.278207+0000","last_became_peered":"2024-09-13T09:29:52.278207+0000","last_unstale":"2024-09-13T09:30:31.012490+0000","last_undegraded":"2024-09-13T09:30:31.012490+0000","last_fullsized":"2024-09-13T09:30:31.012490+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'630","last_scrub_stamp":"2024-09-13T09:30:31.012442+0000","last_deep_scrub":"232'630","last_deep_scrub_stamp":"2024-09-13T09:30:31.012442+0000","last_clean_scrub_stamp":"2024-09-13T09:30:31.012442+0000","objects_scrubbed":1,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:11.524664+0000","scrub_duration":0.0067878080000000002,"objects_trimmed":0,"snaptrim_duration":0.0021856150000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5239,"num_read_kb":24960,"num_write":722,"num_write_kb":13283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":487,"num_bytes_recovered":21696373,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"232'822","reported_seq":6165,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:32.964425+0000","last_change":"2024-09-13T09:30:32.964585+0000","last_active":"2024-09-13T09:30:32.964425+0000","last_peered":"2024-09-13T09:30:32.964425+0000","last_clean":"2024-09-13T09:30:32.964425+0000","last_became_active":"2024-09-13T09:29:52.279341+0000","last_became_peered":"2024-09-13T09:29:52.279341+0000","last_unstale":"2024-09-13T09:30:32.964425+0000","last_undegraded":"2024-09-13T09:30:32.964425+0000","last_fullsized":"2024-09-13T09:30:32.964425+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'822","last_scrub_stamp":"2024-09-13T09:30:32.964394+0000","last_deep_scrub":"232'822","last_deep_scrub_stamp":"2024-09-13T09:30:32.964394+0000","last_clean_scrub_stamp":"2024-09-13T09:30:32.964394+0000","objects_scrubbed":3,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:02:54.222598+0000","scrub_duration":0.010215768,"objects_trimmed":0,"snaptrim_duration":0.0023345729999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":6338,"num_read_kb":19200,"num_write":1130,"num_write_kb":14323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":24639177,"num_keys_recovered":240,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"192'680","reported_seq":7181,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:27.937872+0000","last_change":"2024-09-13T09:30:27.938153+0000","last_active":"2024-09-13T09:30:27.937872+0000","last_peered":"2024-09-13T09:30:27.937872+0000","last_clean":"2024-09-13T09:30:27.937872+0000","last_became_active":"2024-09-13T09:29:16.203226+0000","last_became_peered":"2024-09-13T09:29:16.203226+0000","last_unstale":"2024-09-13T09:30:27.937872+0000","last_undegraded":"2024-09-13T09:30:27.937872+0000","last_fullsized":"2024-09-13T09:30:27.937872+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'680","last_scrub_stamp":"2024-09-13T09:30:27.937816+0000","last_deep_scrub":"192'680","last_deep_scrub_stamp":"2024-09-13T09:30:27.937816+0000","last_clean_scrub_stamp":"2024-09-13T09:30:27.937816+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:30.858036+0000","scrub_duration":0.0038834350000000002,"objects_trimmed":0,"snaptrim_duration":0.0043590440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8009,"num_read_kb":16393,"num_write":697,"num_write_kb":8320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":15467509,"num_keys_recovered":120,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"232'681","reported_seq":13810,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:17.017006+0000","last_change":"2024-09-13T09:30:17.017298+0000","last_active":"2024-09-13T09:30:17.017006+0000","last_peered":"2024-09-13T09:30:17.017006+0000","last_clean":"2024-09-13T09:30:17.017006+0000","last_became_active":"2024-09-13T09:29:13.668593+0000","last_became_peered":"2024-09-13T09:29:13.668593+0000","last_unstale":"2024-09-13T09:30:17.017006+0000","last_undegraded":"2024-09-13T09:30:17.017006+0000","last_fullsized":"2024-09-13T09:30:17.017006+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'681","last_scrub_stamp":"2024-09-13T09:30:17.016955+0000","last_deep_scrub":"232'681","last_deep_scrub_stamp":"2024-09-13T09:30:17.016955+0000","last_clean_scrub_stamp":"2024-09-13T09:30:17.016955+0000","objects_scrubbed":1,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:07:12.805377+0000","scrub_duration":0.0063088859999999997,"objects_trimmed":0,"snaptrim_duration":0.0011013559999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13835,"num_read_kb":24133,"num_write":742,"num_write_kb":9218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":22566264,"num_keys_recovered":115,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"232'668","reported_seq":3883,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:23.864135+0000","last_change":"2024-09-13T09:30:23.864135+0000","last_active":"2024-09-13T09:30:23.864135+0000","last_peered":"2024-09-13T09:30:23.864135+0000","last_clean":"2024-09-13T09:30:23.864135+0000","last_became_active":"2024-09-13T09:27:05.740109+0000","last_became_peered":"2024-09-13T09:27:05.740109+0000","last_unstale":"2024-09-13T09:30:23.864135+0000","last_undegraded":"2024-09-13T09:30:23.864135+0000","last_fullsized":"2024-09-13T09:30:23.864135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'668","last_scrub_stamp":"2024-09-13T09:30:23.863982+0000","last_deep_scrub":"232'668","last_deep_scrub_stamp":"2024-09-13T09:30:23.863982+0000","last_clean_scrub_stamp":"2024-09-13T09:30:23.863982+0000","objects_scrubbed":2,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:22.547564+0000","scrub_duration":0.0086823170000000002,"objects_trimmed":0,"snaptrim_duration":0.0014616290000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4511,"num_read_kb":14977,"num_write":641,"num_write_kb":8153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2339549,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"232'694","reported_seq":5881,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.049996+0000","last_change":"2024-09-13T09:30:20.050220+0000","last_active":"2024-09-13T09:30:20.049996+0000","last_peered":"2024-09-13T09:30:20.049996+0000","last_clean":"2024-09-13T09:30:20.049996+0000","last_became_active":"2024-09-13T09:29:51.280603+0000","last_became_peered":"2024-09-13T09:29:51.280603+0000","last_unstale":"2024-09-13T09:30:20.049996+0000","last_undegraded":"2024-09-13T09:30:20.049996+0000","last_fullsized":"2024-09-13T09:30:20.049996+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'694","last_scrub_stamp":"2024-09-13T09:30:20.049943+0000","last_deep_scrub":"232'694","last_deep_scrub_stamp":"2024-09-13T09:30:20.049943+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.049943+0000","objects_scrubbed":3,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:30.525532+0000","scrub_duration":0.0095315230000000001,"objects_trimmed":0,"snaptrim_duration":0.0064241439999999997,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7507,"num_read_kb":22402,"num_write":656,"num_write_kb":15620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":34484509,"num_keys_recovered":145,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"232'790","reported_seq":3029,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:22.836956+0000","last_change":"2024-09-13T09:30:22.837331+0000","last_active":"2024-09-13T09:30:22.836956+0000","last_peered":"2024-09-13T09:30:22.836956+0000","last_clean":"2024-09-13T09:30:22.836956+0000","last_became_active":"2024-09-13T09:29:54.323584+0000","last_became_peered":"2024-09-13T09:29:54.323584+0000","last_unstale":"2024-09-13T09:30:22.836956+0000","last_undegraded":"2024-09-13T09:30:22.836956+0000","last_fullsized":"2024-09-13T09:30:22.836956+0000","mapping_epoch":219,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":220,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'790","last_scrub_stamp":"2024-09-13T09:30:22.836841+0000","last_deep_scrub":"232'790","last_deep_scrub_stamp":"2024-09-13T09:30:22.836841+0000","last_clean_scrub_stamp":"2024-09-13T09:30:22.836841+0000","objects_scrubbed":2,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:29.200769+0000","scrub_duration":0.0085375950000000003,"objects_trimmed":0,"snaptrim_duration":0.0032784979999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4278,"num_read_kb":9927,"num_write":873,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":538384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"217'605","reported_seq":6178,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:25.026845+0000","last_change":"2024-09-13T09:30:25.027102+0000","last_active":"2024-09-13T09:30:25.026845+0000","last_peered":"2024-09-13T09:30:25.026845+0000","last_clean":"2024-09-13T09:30:25.026845+0000","last_became_active":"2024-09-13T09:29:51.282124+0000","last_became_peered":"2024-09-13T09:29:51.282124+0000","last_unstale":"2024-09-13T09:30:25.026845+0000","last_undegraded":"2024-09-13T09:30:25.026845+0000","last_fullsized":"2024-09-13T09:30:25.026845+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'605","last_scrub_stamp":"2024-09-13T09:30:25.026796+0000","last_deep_scrub":"217'605","last_deep_scrub_stamp":"2024-09-13T09:30:25.026796+0000","last_clean_scrub_stamp":"2024-09-13T09:30:25.026796+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:20.024564+0000","scrub_duration":0.003653281,"objects_trimmed":6,"snaptrim_duration":0.022701341999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5818,"num_read_kb":36062,"num_write":688,"num_write_kb":20374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"232'498","reported_seq":10601,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:30.744599+0000","last_change":"2024-09-13T09:30:30.744599+0000","last_active":"2024-09-13T09:30:30.744599+0000","last_peered":"2024-09-13T09:30:30.744599+0000","last_clean":"2024-09-13T09:30:30.744599+0000","last_became_active":"2024-09-13T09:27:05.749136+0000","last_became_peered":"2024-09-13T09:27:05.749136+0000","last_unstale":"2024-09-13T09:30:30.744599+0000","last_undegraded":"2024-09-13T09:30:30.744599+0000","last_fullsized":"2024-09-13T09:30:30.744599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'498","last_scrub_stamp":"2024-09-13T09:30:30.744316+0000","last_deep_scrub":"232'498","last_deep_scrub_stamp":"2024-09-13T09:30:30.744316+0000","last_clean_scrub_stamp":"2024-09-13T09:30:30.744316+0000","objects_scrubbed":1,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:02:27.863929+0000","scrub_duration":0.0069570109999999999,"objects_trimmed":0,"snaptrim_duration":0.0010032649999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":10266,"num_read_kb":33265,"num_write":497,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":2103586,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"232'614","reported_seq":6532,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:24.056515+0000","last_change":"2024-09-13T09:30:24.056787+0000","last_active":"2024-09-13T09:30:24.056515+0000","last_peered":"2024-09-13T09:30:24.056515+0000","last_clean":"2024-09-13T09:30:24.056515+0000","last_became_active":"2024-09-13T09:29:14.681749+0000","last_became_peered":"2024-09-13T09:29:14.681749+0000","last_unstale":"2024-09-13T09:30:24.056515+0000","last_undegraded":"2024-09-13T09:30:24.056515+0000","last_fullsized":"2024-09-13T09:30:24.056515+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'614","last_scrub_stamp":"2024-09-13T09:30:24.056465+0000","last_deep_scrub":"232'614","last_deep_scrub_stamp":"2024-09-13T09:30:24.056465+0000","last_clean_scrub_stamp":"2024-09-13T09:30:24.056465+0000","objects_scrubbed":1,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:25.566240+0000","scrub_duration":0.0075038919999999999,"objects_trimmed":0,"snaptrim_duration":0.0015383829999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6505,"num_read_kb":13666,"num_write":654,"num_write_kb":6159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":8018429,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":252,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:17.667463+0000","last_change":"2024-09-13T09:30:17.667797+0000","last_active":"2024-09-13T09:30:17.667463+0000","last_peered":"2024-09-13T09:30:17.667463+0000","last_clean":"2024-09-13T09:30:17.667463+0000","last_became_active":"2024-09-13T09:27:02.704087+0000","last_became_peered":"2024-09-13T09:27:02.704087+0000","last_unstale":"2024-09-13T09:30:17.667463+0000","last_undegraded":"2024-09-13T09:30:17.667463+0000","last_fullsized":"2024-09-13T09:30:17.667463+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:17.667384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:17.667384+0000","last_clean_scrub_stamp":"2024-09-13T09:30:17.667384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:49.179677+0000","scrub_duration":0.0037910259999999999,"objects_trimmed":0,"snaptrim_duration":0.001151053,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"232'541","reported_seq":2922,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:24.038595+0000","last_change":"2024-09-13T09:30:24.038843+0000","last_active":"2024-09-13T09:30:24.038595+0000","last_peered":"2024-09-13T09:30:24.038595+0000","last_clean":"2024-09-13T09:30:24.038595+0000","last_became_active":"2024-09-13T09:29:51.284873+0000","last_became_peered":"2024-09-13T09:29:51.284873+0000","last_unstale":"2024-09-13T09:30:24.038595+0000","last_undegraded":"2024-09-13T09:30:24.038595+0000","last_fullsized":"2024-09-13T09:30:24.038595+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'541","last_scrub_stamp":"2024-09-13T09:30:24.038543+0000","last_deep_scrub":"232'541","last_deep_scrub_stamp":"2024-09-13T09:30:24.038543+0000","last_clean_scrub_stamp":"2024-09-13T09:30:24.038543+0000","objects_scrubbed":1,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:58.883135+0000","scrub_duration":0.0075833929999999999,"objects_trimmed":4,"snaptrim_duration":0.015847012000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":2311,"num_read_kb":6501,"num_write":582,"num_write_kb":8111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":1475638,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":260,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.628611+0000","last_change":"2024-09-13T09:30:18.628981+0000","last_active":"2024-09-13T09:30:18.628611+0000","last_peered":"2024-09-13T09:30:18.628611+0000","last_clean":"2024-09-13T09:30:18.628611+0000","last_became_active":"2024-09-13T09:26:39.539957+0000","last_became_peered":"2024-09-13T09:26:39.539957+0000","last_unstale":"2024-09-13T09:30:18.628611+0000","last_undegraded":"2024-09-13T09:30:18.628611+0000","last_fullsized":"2024-09-13T09:30:18.628611+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:18.628521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:18.628521+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.628521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:29:27.378445+0000","scrub_duration":0.0038613499999999999,"objects_trimmed":0,"snaptrim_duration":0.001262113,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"232'966","reported_seq":4822,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.912256+0000","last_change":"2024-09-13T09:30:21.912588+0000","last_active":"2024-09-13T09:30:21.912256+0000","last_peered":"2024-09-13T09:30:21.912256+0000","last_clean":"2024-09-13T09:30:21.912256+0000","last_became_active":"2024-09-13T09:29:40.200432+0000","last_became_peered":"2024-09-13T09:29:40.200432+0000","last_unstale":"2024-09-13T09:30:21.912256+0000","last_undegraded":"2024-09-13T09:30:21.912256+0000","last_fullsized":"2024-09-13T09:30:21.912256+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'966","last_scrub_stamp":"2024-09-13T09:30:21.912207+0000","last_deep_scrub":"232'966","last_deep_scrub_stamp":"2024-09-13T09:30:21.912207+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.912207+0000","objects_scrubbed":1,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:34.233136+0000","scrub_duration":0.0077823739999999999,"objects_trimmed":0,"snaptrim_duration":0.0013878059999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6394,"num_read_kb":19902,"num_write":1236,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":11658430,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.006475+0000","last_change":"2024-09-13T09:30:18.006732+0000","last_active":"2024-09-13T09:30:18.006475+0000","last_peered":"2024-09-13T09:30:18.006475+0000","last_clean":"2024-09-13T09:30:18.006475+0000","last_became_active":"2024-09-13T09:29:40.201035+0000","last_became_peered":"2024-09-13T09:29:40.201035+0000","last_unstale":"2024-09-13T09:30:18.006475+0000","last_undegraded":"2024-09-13T09:30:18.006475+0000","last_fullsized":"2024-09-13T09:30:18.006475+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:18.006431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:18.006431+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.006431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:26:42.425851+0000","scrub_duration":0.003406259,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"232'881","reported_seq":9203,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:49.026292+0000","last_change":"2024-09-13T09:30:49.026527+0000","last_active":"2024-09-13T09:30:49.026292+0000","last_peered":"2024-09-13T09:30:49.026292+0000","last_clean":"2024-09-13T09:30:49.026292+0000","last_became_active":"2024-09-13T09:27:05.740447+0000","last_became_peered":"2024-09-13T09:27:05.740447+0000","last_unstale":"2024-09-13T09:30:49.026292+0000","last_undegraded":"2024-09-13T09:30:49.026292+0000","last_fullsized":"2024-09-13T09:30:49.026292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'881","last_scrub_stamp":"2024-09-13T09:30:49.026234+0000","last_deep_scrub":"232'881","last_deep_scrub_stamp":"2024-09-13T09:30:49.026234+0000","last_clean_scrub_stamp":"2024-09-13T09:30:49.026234+0000","objects_scrubbed":1,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:19:23.054740+0000","scrub_duration":0.0082432559999999992,"objects_trimmed":0,"snaptrim_duration":0.002251547,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11185,"num_read_kb":32348,"num_write":1131,"num_write_kb":11247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":1587153,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":212,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:16.859042+0000","last_change":"2024-09-13T09:30:16.859146+0000","last_active":"2024-09-13T09:30:16.859042+0000","last_peered":"2024-09-13T09:30:16.859042+0000","last_clean":"2024-09-13T09:30:16.859042+0000","last_became_active":"2024-09-13T09:29:51.280884+0000","last_became_peered":"2024-09-13T09:29:51.280884+0000","last_unstale":"2024-09-13T09:30:16.859042+0000","last_undegraded":"2024-09-13T09:30:16.859042+0000","last_fullsized":"2024-09-13T09:30:16.859042+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:16.859023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:16.859023+0000","last_clean_scrub_stamp":"2024-09-13T09:30:16.859023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:30:39.958454+0000","scrub_duration":0.002657745,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"232'930","reported_seq":6289,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:25.876119+0000","last_change":"2024-09-13T09:30:25.876395+0000","last_active":"2024-09-13T09:30:25.876119+0000","last_peered":"2024-09-13T09:30:25.876119+0000","last_clean":"2024-09-13T09:30:25.876119+0000","last_became_active":"2024-09-13T09:26:42.568887+0000","last_became_peered":"2024-09-13T09:26:42.568887+0000","last_unstale":"2024-09-13T09:30:25.876119+0000","last_undegraded":"2024-09-13T09:30:25.876119+0000","last_fullsized":"2024-09-13T09:30:25.876119+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'930","last_scrub_stamp":"2024-09-13T09:30:25.876059+0000","last_deep_scrub":"232'930","last_deep_scrub_stamp":"2024-09-13T09:30:25.876059+0000","last_clean_scrub_stamp":"2024-09-13T09:30:25.876059+0000","objects_scrubbed":2,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:45.712490+0000","scrub_duration":0.010502780999999999,"objects_trimmed":0,"snaptrim_duration":0.0039543720000000003,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":8427,"num_read_kb":25244,"num_write":1056,"num_write_kb":10493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":1261636,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"232'1427","reported_seq":4493,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:32.992829+0000","last_change":"2024-09-13T09:30:32.992829+0000","last_active":"2024-09-13T09:30:32.992829+0000","last_peered":"2024-09-13T09:30:32.992829+0000","last_clean":"2024-09-13T09:30:32.992829+0000","last_became_active":"2024-09-13T09:29:15.192634+0000","last_became_peered":"2024-09-13T09:29:15.192634+0000","last_unstale":"2024-09-13T09:30:32.992829+0000","last_undegraded":"2024-09-13T09:30:32.992829+0000","last_fullsized":"2024-09-13T09:30:32.992829+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'1427","last_scrub_stamp":"2024-09-13T09:30:32.992359+0000","last_deep_scrub":"232'1427","last_deep_scrub_stamp":"2024-09-13T09:30:32.992359+0000","last_clean_scrub_stamp":"2024-09-13T09:30:32.992359+0000","objects_scrubbed":3,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:38.667370+0000","scrub_duration":0.010444729,"objects_trimmed":0,"snaptrim_duration":0.001761573,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":8088,"num_read_kb":20355,"num_write":2096,"num_write_kb":7035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":415,"num_bytes_recovered":10575445,"num_keys_recovered":140,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"232'119","reported_seq":385,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.813403+0000","last_change":"2024-09-13T09:30:20.813706+0000","last_active":"2024-09-13T09:30:20.813403+0000","last_peered":"2024-09-13T09:30:20.813403+0000","last_clean":"2024-09-13T09:30:20.813403+0000","last_became_active":"2024-09-13T09:25:43.997404+0000","last_became_peered":"2024-09-13T09:25:43.997404+0000","last_unstale":"2024-09-13T09:30:20.813403+0000","last_undegraded":"2024-09-13T09:30:20.813403+0000","last_fullsized":"2024-09-13T09:30:20.813403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'119","last_scrub_stamp":"2024-09-13T09:30:20.813350+0000","last_deep_scrub":"232'119","last_deep_scrub_stamp":"2024-09-13T09:30:20.813350+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.813350+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:26:53.848530+0000","scrub_duration":0.010415627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"232'2","reported_seq":242,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.872171+0000","last_change":"2024-09-13T09:30:18.872412+0000","last_active":"2024-09-13T09:30:18.872171+0000","last_peered":"2024-09-13T09:30:18.872171+0000","last_clean":"2024-09-13T09:30:18.872171+0000","last_became_active":"2024-09-13T09:25:42.951633+0000","last_became_peered":"2024-09-13T09:25:42.951633+0000","last_unstale":"2024-09-13T09:30:18.872171+0000","last_undegraded":"2024-09-13T09:30:18.872171+0000","last_fullsized":"2024-09-13T09:30:18.872171+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'2","last_scrub_stamp":"2024-09-13T09:30:18.872122+0000","last_deep_scrub":"232'2","last_deep_scrub_stamp":"2024-09-13T09:30:18.872122+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.872122+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:06:16.029753+0000","scrub_duration":0.006480754,"objects_trimmed":0,"snaptrim_duration":0.00041206000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"211'1021","reported_seq":9550,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:26.880365+0000","last_change":"2024-09-13T09:30:26.880744+0000","last_active":"2024-09-13T09:30:26.880365+0000","last_peered":"2024-09-13T09:30:26.880365+0000","last_clean":"2024-09-13T09:30:26.880365+0000","last_became_active":"2024-09-13T09:29:40.202441+0000","last_became_peered":"2024-09-13T09:29:40.202441+0000","last_unstale":"2024-09-13T09:30:26.880365+0000","last_undegraded":"2024-09-13T09:30:26.880365+0000","last_fullsized":"2024-09-13T09:30:26.880365+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'1021","last_scrub_stamp":"2024-09-13T09:30:26.880257+0000","last_deep_scrub":"211'1021","last_deep_scrub_stamp":"2024-09-13T09:30:26.880257+0000","last_clean_scrub_stamp":"2024-09-13T09:30:26.880257+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:38:15.127399+0000","scrub_duration":0.0038710429999999998,"objects_trimmed":0,"snaptrim_duration":0.00078854900000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11008,"num_read_kb":23531,"num_write":1430,"num_write_kb":18970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":11660557,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"232'3","reported_seq":266,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:18.809480+0000","last_change":"2024-09-13T09:30:18.809736+0000","last_active":"2024-09-13T09:30:18.809480+0000","last_peered":"2024-09-13T09:30:18.809480+0000","last_clean":"2024-09-13T09:30:18.809480+0000","last_became_active":"2024-09-13T09:26:39.541055+0000","last_became_peered":"2024-09-13T09:26:39.541055+0000","last_unstale":"2024-09-13T09:30:18.809480+0000","last_undegraded":"2024-09-13T09:30:18.809480+0000","last_fullsized":"2024-09-13T09:30:18.809480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'3","last_scrub_stamp":"2024-09-13T09:30:18.809433+0000","last_deep_scrub":"232'3","last_deep_scrub_stamp":"2024-09-13T09:30:18.809433+0000","last_clean_scrub_stamp":"2024-09-13T09:30:18.809433+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:44:22.407760+0000","scrub_duration":0.006879736,"objects_trimmed":0,"snaptrim_duration":0.001583662,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"232'843","reported_seq":11382,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:27.126537+0000","last_change":"2024-09-13T09:30:27.126833+0000","last_active":"2024-09-13T09:30:27.126537+0000","last_peered":"2024-09-13T09:30:27.126537+0000","last_clean":"2024-09-13T09:30:27.126537+0000","last_became_active":"2024-09-13T09:29:13.665303+0000","last_became_peered":"2024-09-13T09:29:13.665303+0000","last_unstale":"2024-09-13T09:30:27.126537+0000","last_undegraded":"2024-09-13T09:30:27.126537+0000","last_fullsized":"2024-09-13T09:30:27.126537+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'843","last_scrub_stamp":"2024-09-13T09:30:27.126466+0000","last_deep_scrub":"232'843","last_deep_scrub_stamp":"2024-09-13T09:30:27.126466+0000","last_clean_scrub_stamp":"2024-09-13T09:30:27.126466+0000","objects_scrubbed":3,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:33.427533+0000","scrub_duration":0.010701713999999999,"objects_trimmed":0,"snaptrim_duration":0.0016356770000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":10107,"num_read_kb":39049,"num_write":924,"num_write_kb":31820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":52837303,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":194,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.066262+0000","last_change":"2024-09-13T09:30:21.066461+0000","last_active":"2024-09-13T09:30:21.066262+0000","last_peered":"2024-09-13T09:30:21.066262+0000","last_clean":"2024-09-13T09:30:21.066262+0000","last_became_active":"2024-09-13T09:29:51.279138+0000","last_became_peered":"2024-09-13T09:29:51.279138+0000","last_unstale":"2024-09-13T09:30:21.066262+0000","last_undegraded":"2024-09-13T09:30:21.066262+0000","last_fullsized":"2024-09-13T09:30:21.066262+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:21.066217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:21.066217+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.066217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:43.942714+0000","scrub_duration":0.0035654110000000001,"objects_trimmed":0,"snaptrim_duration":0.001042039,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"232'2105","reported_seq":7114,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:40.065482+0000","last_change":"2024-09-13T09:30:40.065742+0000","last_active":"2024-09-13T09:30:40.065482+0000","last_peered":"2024-09-13T09:30:40.065482+0000","last_clean":"2024-09-13T09:30:40.065482+0000","last_became_active":"2024-09-13T09:29:51.284196+0000","last_became_peered":"2024-09-13T09:29:51.284196+0000","last_unstale":"2024-09-13T09:30:40.065482+0000","last_undegraded":"2024-09-13T09:30:40.065482+0000","last_fullsized":"2024-09-13T09:30:40.065482+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'2105","last_scrub_stamp":"2024-09-13T09:30:40.065430+0000","last_deep_scrub":"232'2105","last_deep_scrub_stamp":"2024-09-13T09:30:40.065430+0000","last_clean_scrub_stamp":"2024-09-13T09:30:40.065430+0000","objects_scrubbed":1,"log_size":2105,"log_dups_size":0,"ondisk_log_size":2105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:56:23.204606+0000","scrub_duration":0.0085094969999999995,"objects_trimmed":2,"snaptrim_duration":0.011054130000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":14701,"num_read_kb":17092,"num_write":3721,"num_write_kb":7314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":50,"num_bytes_recovered":938847,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":255,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:19.843861+0000","last_change":"2024-09-13T09:30:19.844112+0000","last_active":"2024-09-13T09:30:19.843861+0000","last_peered":"2024-09-13T09:30:19.843861+0000","last_clean":"2024-09-13T09:30:19.843861+0000","last_became_active":"2024-09-13T09:26:39.540141+0000","last_became_peered":"2024-09-13T09:26:39.540141+0000","last_unstale":"2024-09-13T09:30:19.843861+0000","last_undegraded":"2024-09-13T09:30:19.843861+0000","last_fullsized":"2024-09-13T09:30:19.843861+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:30:19.843817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:30:19.843817+0000","last_clean_scrub_stamp":"2024-09-13T09:30:19.843817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:37:47.416004+0000","scrub_duration":0.0035090439999999998,"objects_trimmed":0,"snaptrim_duration":0.0024415470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"232'654","reported_seq":3518,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:22.850990+0000","last_change":"2024-09-13T09:30:22.851306+0000","last_active":"2024-09-13T09:30:22.850990+0000","last_peered":"2024-09-13T09:30:22.850990+0000","last_clean":"2024-09-13T09:30:22.850990+0000","last_became_active":"2024-09-13T09:27:05.744152+0000","last_became_peered":"2024-09-13T09:27:05.744152+0000","last_unstale":"2024-09-13T09:30:22.850990+0000","last_undegraded":"2024-09-13T09:30:22.850990+0000","last_fullsized":"2024-09-13T09:30:22.850990+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'654","last_scrub_stamp":"2024-09-13T09:30:22.850943+0000","last_deep_scrub":"232'654","last_deep_scrub_stamp":"2024-09-13T09:30:22.850943+0000","last_clean_scrub_stamp":"2024-09-13T09:30:22.850943+0000","objects_scrubbed":1,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:20.209409+0000","scrub_duration":0.0071269209999999996,"objects_trimmed":0,"snaptrim_duration":0.0014274909999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4411,"num_read_kb":15168,"num_write":670,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":883582,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"232'488","reported_seq":12066,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.072775+0000","last_change":"2024-09-13T09:30:21.073021+0000","last_active":"2024-09-13T09:30:21.072775+0000","last_peered":"2024-09-13T09:30:21.072775+0000","last_clean":"2024-09-13T09:30:21.072775+0000","last_became_active":"2024-09-13T09:29:40.201154+0000","last_became_peered":"2024-09-13T09:29:40.201154+0000","last_unstale":"2024-09-13T09:30:21.072775+0000","last_undegraded":"2024-09-13T09:30:21.072775+0000","last_fullsized":"2024-09-13T09:30:21.072775+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'488","last_scrub_stamp":"2024-09-13T09:30:21.072709+0000","last_deep_scrub":"232'488","last_deep_scrub_stamp":"2024-09-13T09:30:21.072709+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.072709+0000","objects_scrubbed":5,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:49.706218+0000","scrub_duration":0.011994694,"objects_trimmed":0,"snaptrim_duration":0.0011244759999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":10498,"num_read_kb":27484,"num_write":485,"num_write_kb":18976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":760,"num_bytes_recovered":25872534,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"232'574","reported_seq":7664,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.719028+0000","last_change":"2024-09-13T09:30:21.719342+0000","last_active":"2024-09-13T09:30:21.719028+0000","last_peered":"2024-09-13T09:30:21.719028+0000","last_clean":"2024-09-13T09:30:21.719028+0000","last_became_active":"2024-09-13T09:29:15.194028+0000","last_became_peered":"2024-09-13T09:29:15.194028+0000","last_unstale":"2024-09-13T09:30:21.719028+0000","last_undegraded":"2024-09-13T09:30:21.719028+0000","last_fullsized":"2024-09-13T09:30:21.719028+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'574","last_scrub_stamp":"2024-09-13T09:30:21.718979+0000","last_deep_scrub":"232'574","last_deep_scrub_stamp":"2024-09-13T09:30:21.718979+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.718979+0000","objects_scrubbed":2,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:32:07.720115+0000","scrub_duration":0.0080011579999999995,"objects_trimmed":0,"snaptrim_duration":0.0015087329999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":6489,"num_read_kb":26578,"num_write":546,"num_write_kb":18947,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":412,"num_bytes_recovered":13242582,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"232'623","reported_seq":4519,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.869500+0000","last_change":"2024-09-13T09:30:20.869822+0000","last_active":"2024-09-13T09:30:20.869500+0000","last_peered":"2024-09-13T09:30:20.869500+0000","last_clean":"2024-09-13T09:30:20.869500+0000","last_became_active":"2024-09-13T09:27:05.739224+0000","last_became_peered":"2024-09-13T09:27:05.739224+0000","last_unstale":"2024-09-13T09:30:20.869500+0000","last_undegraded":"2024-09-13T09:30:20.869500+0000","last_fullsized":"2024-09-13T09:30:20.869500+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'623","last_scrub_stamp":"2024-09-13T09:30:20.869450+0000","last_deep_scrub":"232'623","last_deep_scrub_stamp":"2024-09-13T09:30:20.869450+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.869450+0000","objects_scrubbed":1,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:29:30.985753+0000","scrub_duration":0.0067024739999999999,"objects_trimmed":0,"snaptrim_duration":0.0017830439999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3853,"num_read_kb":28425,"num_write":686,"num_write_kb":20453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":5274003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"217'596","reported_seq":7652,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:29.014610+0000","last_change":"2024-09-13T09:30:29.014861+0000","last_active":"2024-09-13T09:30:29.014610+0000","last_peered":"2024-09-13T09:30:29.014610+0000","last_clean":"2024-09-13T09:30:29.014610+0000","last_became_active":"2024-09-13T09:29:51.284911+0000","last_became_peered":"2024-09-13T09:29:51.284911+0000","last_unstale":"2024-09-13T09:30:29.014610+0000","last_undegraded":"2024-09-13T09:30:29.014610+0000","last_fullsized":"2024-09-13T09:30:29.014610+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'596","last_scrub_stamp":"2024-09-13T09:30:29.014557+0000","last_deep_scrub":"217'596","last_deep_scrub_stamp":"2024-09-13T09:30:29.014557+0000","last_clean_scrub_stamp":"2024-09-13T09:30:29.014557+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:13.992077+0000","scrub_duration":0.0038990359999999998,"objects_trimmed":2,"snaptrim_duration":0.016126896000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7761,"num_read_kb":22443,"num_write":801,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":4939467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"232'500","reported_seq":14681,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:21.776122+0000","last_change":"2024-09-13T09:30:21.776122+0000","last_active":"2024-09-13T09:30:21.776122+0000","last_peered":"2024-09-13T09:30:21.776122+0000","last_clean":"2024-09-13T09:30:21.776122+0000","last_became_active":"2024-09-13T09:29:51.285108+0000","last_became_peered":"2024-09-13T09:29:51.285108+0000","last_unstale":"2024-09-13T09:30:21.776122+0000","last_undegraded":"2024-09-13T09:30:21.776122+0000","last_fullsized":"2024-09-13T09:30:21.776122+0000","mapping_epoch":216,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":217,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'500","last_scrub_stamp":"2024-09-13T09:30:21.775794+0000","last_deep_scrub":"232'500","last_deep_scrub_stamp":"2024-09-13T09:30:21.775794+0000","last_clean_scrub_stamp":"2024-09-13T09:30:21.775794+0000","objects_scrubbed":2,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:30.525491+0000","scrub_duration":0.007365595,"objects_trimmed":6,"snaptrim_duration":0.015559683,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":13905,"num_read_kb":36195,"num_write":543,"num_write_kb":30956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"232'1207","reported_seq":9174,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:27.719680+0000","last_change":"2024-09-13T09:30:27.719814+0000","last_active":"2024-09-13T09:30:27.719680+0000","last_peered":"2024-09-13T09:30:27.719680+0000","last_clean":"2024-09-13T09:30:27.719680+0000","last_became_active":"2024-09-13T09:29:15.191379+0000","last_became_peered":"2024-09-13T09:29:15.191379+0000","last_unstale":"2024-09-13T09:30:27.719680+0000","last_undegraded":"2024-09-13T09:30:27.719680+0000","last_fullsized":"2024-09-13T09:30:27.719680+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'1207","last_scrub_stamp":"2024-09-13T09:30:27.719652+0000","last_deep_scrub":"232'1207","last_deep_scrub_stamp":"2024-09-13T09:30:27.719652+0000","last_clean_scrub_stamp":"2024-09-13T09:30:27.719652+0000","objects_scrubbed":2,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:51:53.584551+0000","scrub_duration":0.0088464070000000006,"objects_trimmed":0,"snaptrim_duration":0.0032583339999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":11018,"num_read_kb":24057,"num_write":1830,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":21552348,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"232'484","reported_seq":7279,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:29.786664+0000","last_change":"2024-09-13T09:30:29.786879+0000","last_active":"2024-09-13T09:30:29.786664+0000","last_peered":"2024-09-13T09:30:29.786664+0000","last_clean":"2024-09-13T09:30:29.786664+0000","last_became_active":"2024-09-13T09:29:12.662804+0000","last_became_peered":"2024-09-13T09:29:12.662804+0000","last_unstale":"2024-09-13T09:30:29.786664+0000","last_undegraded":"2024-09-13T09:30:29.786664+0000","last_fullsized":"2024-09-13T09:30:29.786664+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'484","last_scrub_stamp":"2024-09-13T09:30:29.786616+0000","last_deep_scrub":"232'484","last_deep_scrub_stamp":"2024-09-13T09:30:29.786616+0000","last_clean_scrub_stamp":"2024-09-13T09:30:29.786616+0000","objects_scrubbed":1,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:10.267279+0000","scrub_duration":0.0066261050000000002,"objects_trimmed":0,"snaptrim_duration":0.0033360640000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5651,"num_read_kb":30408,"num_write":506,"num_write_kb":14303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":446,"num_bytes_recovered":21349680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"232'888","reported_seq":4084,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:20.680788+0000","last_change":"2024-09-13T09:30:20.680788+0000","last_active":"2024-09-13T09:30:20.680788+0000","last_peered":"2024-09-13T09:30:20.680788+0000","last_clean":"2024-09-13T09:30:20.680788+0000","last_became_active":"2024-09-13T09:26:42.573306+0000","last_became_peered":"2024-09-13T09:26:42.573306+0000","last_unstale":"2024-09-13T09:30:20.680788+0000","last_undegraded":"2024-09-13T09:30:20.680788+0000","last_fullsized":"2024-09-13T09:30:20.680788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'888","last_scrub_stamp":"2024-09-13T09:30:20.680505+0000","last_deep_scrub":"232'888","last_deep_scrub_stamp":"2024-09-13T09:30:20.680505+0000","last_clean_scrub_stamp":"2024-09-13T09:30:20.680505+0000","objects_scrubbed":1,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:13.433630+0000","scrub_duration":0.0089656230000000007,"objects_trimmed":0,"snaptrim_duration":0.0014993859999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4992,"num_read_kb":24585,"num_write":1208,"num_write_kb":10616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":3170851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"232'897","reported_seq":9220,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:22.774863+0000","last_change":"2024-09-13T09:30:22.774863+0000","last_active":"2024-09-13T09:30:22.774863+0000","last_peered":"2024-09-13T09:30:22.774863+0000","last_clean":"2024-09-13T09:30:22.774863+0000","last_became_active":"2024-09-13T09:29:16.203391+0000","last_became_peered":"2024-09-13T09:29:16.203391+0000","last_unstale":"2024-09-13T09:30:22.774863+0000","last_undegraded":"2024-09-13T09:30:22.774863+0000","last_fullsized":"2024-09-13T09:30:22.774863+0000","mapping_epoch":197,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":198,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'897","last_scrub_stamp":"2024-09-13T09:30:22.774515+0000","last_deep_scrub":"232'897","last_deep_scrub_stamp":"2024-09-13T09:30:22.774515+0000","last_clean_scrub_stamp":"2024-09-13T09:30:22.774515+0000","objects_scrubbed":2,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:32.820000+0000","scrub_duration":0.0083601460000000006,"objects_trimmed":0,"snaptrim_duration":0.0032146150000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":10979,"num_read_kb":26469,"num_write":1037,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":411,"num_bytes_recovered":17279595,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"232'592","reported_seq":6254,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:36.989751+0000","last_change":"2024-09-13T09:30:36.989987+0000","last_active":"2024-09-13T09:30:36.989751+0000","last_peered":"2024-09-13T09:30:36.989751+0000","last_clean":"2024-09-13T09:30:36.989751+0000","last_became_active":"2024-09-13T09:27:17.865917+0000","last_became_peered":"2024-09-13T09:27:17.865917+0000","last_unstale":"2024-09-13T09:30:36.989751+0000","last_undegraded":"2024-09-13T09:30:36.989751+0000","last_fullsized":"2024-09-13T09:30:36.989751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'592","last_scrub_stamp":"2024-09-13T09:30:36.989700+0000","last_deep_scrub":"232'592","last_deep_scrub_stamp":"2024-09-13T09:30:36.989700+0000","last_clean_scrub_stamp":"2024-09-13T09:30:36.989700+0000","objects_scrubbed":1,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:08:13.080290+0000","scrub_duration":0.008070509,"objects_trimmed":0,"snaptrim_duration":0.0016764060000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5907,"num_read_kb":27027,"num_write":700,"num_write_kb":15484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":19682896,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"232'1123","reported_seq":9808,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-13T09:30:28.743148+0000","last_change":"2024-09-13T09:30:28.743148+0000","last_active":"2024-09-13T09:30:28.743148+0000","last_peered":"2024-09-13T09:30:28.743148+0000","last_clean":"2024-09-13T09:30:28.743148+0000","last_became_active":"2024-09-13T09:29:14.679792+0000","last_became_peered":"2024-09-13T09:29:14.679792+0000","last_unstale":"2024-09-13T09:30:28.743148+0000","last_undegraded":"2024-09-13T09:30:28.743148+0000","last_fullsized":"2024-09-13T09:30:28.743148+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'1123","last_scrub_stamp":"2024-09-13T09:30:28.742803+0000","last_deep_scrub":"232'1123","last_deep_scrub_stamp":"2024-09-13T09:30:28.742803+0000","last_clean_scrub_stamp":"2024-09-13T09:30:28.742803+0000","objects_scrubbed":1,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:06:37.618714+0000","scrub_duration":0.0072346880000000004,"objects_trimmed":0,"snaptrim_duration":0.0026145769999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":11298,"num_read_kb":35254,"num_write":1390,"num_write_kb":11792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":811,"num_bytes_recovered":28665708,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":44,"num_read":251281,"num_read_kb":753807,"num_write":31529,"num_write_kb":439053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8886,"num_bytes_recovered":387711939,"num_keys_recovered":1263,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":58709,"internal_metadata":0},"log_size":25391,"ondisk_log_size":25391,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66014,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51964,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94319876,"statfs":{"total":96636764160,"available":96583553024,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76100000000000001,"15min":0.77700000000000002},"min":{"1min":0.59899999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.72899999999999998,"15min":0.76800000000000002},"min":{"1min":0.48799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.97299999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.81299999999999994,"15min":0.80200000000000005},"min":{"1min":0.66500000000000004,"5min":0.375,"15min":0.375},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.105},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.999}]},{"osd":3,"last update":"Fri Sep 13 09:30:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":1.054,"15min":0.97899999999999998},"min":{"1min":0.63800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.052,"5min":13.1,"15min":13.1},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81899999999999995,"15min":0.82499999999999996},"min":{"1min":0.54100000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:30:51 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":0.85099999999999998,"15min":0.80700000000000005},"min":{"1min":0.73499999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.90700000000000003,"15min":0.879},"min":{"1min":0.81599999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.1579999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:30:51 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.135},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":1.204}]},{"osd":6,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.94999999999999996,"15min":1.0349999999999999},"min":{"1min":0.78100000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.202,"5min":1.496,"15min":1.496},"last":1.044},{"interface":"front","average":{"1min":1.1739999999999999,"5min":0.98499999999999999,"15min":1.0149999999999999},"min":{"1min":1.0309999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.4670000000000001,"5min":1.573,"15min":1.573},"last":1.181}]}]},{"osd":6,"up_from":15,"seq":64424509507,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55992,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315848,"statfs":{"total":96636764160,"available":96579428352,"internally_reserved":0,"allocated":2801664,"data_stored":1358391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78200000000000003,"15min":0.86799999999999999},"min":{"1min":0.41399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.506,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.79700000000000004,"15min":0.73099999999999998},"min":{"1min":0.63,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.621,"5min":5.2199999999999998,"15min":5.2199999999999998},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.86199999999999999,"15min":0.94299999999999995},"min":{"1min":0.48499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.292,"5min":5.0339999999999998,"15min":5.0339999999999998},"last":1.119},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.81200000000000006,"5min":0.371,"15min":0.371},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:30:45 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.381},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:30:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55300000000000005,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"last":1.05},{"interface":"front","average":{"1min":0.753,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.77300000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.84599999999999997,"15min":0.83199999999999996},"min":{"1min":0.67400000000000004,"5min":0.436,"15min":0.436},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.292},{"interface":"front","average":{"1min":1.052,"5min":0.89600000000000002,"15min":0.87},"min":{"1min":0.54200000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.95599999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.90500000000000003,"15min":0.90300000000000002},"min":{"1min":0.65000000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.196,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.196},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0389999999999999,"15min":1.2969999999999999},"min":{"1min":0.68799999999999994,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.2130000000000001,"5min":6.2050000000000001,"15min":6.2050000000000001},"last":0.89200000000000002}]}]},{"osd":5,"up_from":216,"seq":927712935951,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":2728,"kb_used_omap":28,"kb_used_meta":29347,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":2793472,"data_stored":1356285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29225,"internal_metadata":30051799},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.95499999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.075}]},{"osd":2,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.0029999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.91400000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.879}]},{"osd":7,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0529999999999999}]}]},{"osd":4,"up_from":173,"seq":743029342236,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":2736,"kb_used_omap":40,"kb_used_meta":29015,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":41191,"internal_metadata":29712153},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.80900000000000005,"15min":0.75700000000000001},"min":{"1min":0.60499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.85599999999999998,"15min":0.84299999999999997},"min":{"1min":0.66900000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2949999999999999,"5min":1.306,"15min":1.306},"last":0.80800000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":0.81299999999999994,"15min":0.76900000000000002},"min":{"1min":0.69899999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.88300000000000001,"15min":0.85299999999999998},"min":{"1min":0.82899999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.294,"5min":1.52,"15min":1.52},"last":1.206}]},{"osd":2,"last update":"Fri Sep 13 09:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.89600000000000002,"15min":0.875},"min":{"1min":0.70999999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.28,"5min":1.339,"15min":1.339},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.96499999999999997,"15min":0.94399999999999995},"min":{"1min":0.71699999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.024},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.84799999999999998,"15min":0.83299999999999996},"min":{"1min":0.747,"5min":0.443,"15min":0.443},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":0.97999999999999998,"15min":0.93799999999999994},"min":{"1min":0.91100000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.099}]},{"osd":7,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":0.98099999999999998,"15min":0.95499999999999996},"min":{"1min":0.83499999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.534,"5min":1.593,"15min":1.593},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.98899999999999999,"15min":0.97999999999999998},"min":{"1min":0.88200000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1}]}]},{"osd":3,"up_from":74,"seq":317827579953,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79920,"kb_used_data":2728,"kb_used_omap":14,"kb_used_meta":77169,"kb_avail":94291920,"statfs":{"total":96636764160,"available":96554926080,"internally_reserved":0,"allocated":2793472,"data_stored":1356644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14794,"internal_metadata":79021622},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65300000000000002,"15min":0.66000000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.1679999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.88700000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72899999999999998},"min":{"1min":0.44,"5min":0.371,"15min":0.371},"max":{"1min":1.204,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.101,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.97399999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.90600000000000003,"15min":0.89800000000000002},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.423,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.026,"15min":1.0229999999999999},"min":{"1min":0.77000000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.4750000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.151}]},{"osd":5,"last update":"Fri Sep 13 09:30:52 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.4039999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":1.002,"15min":1.032},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.5269999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.379},{"interface":"front","average":{"1min":0.999,"5min":1.016,"15min":1.0229999999999999},"min":{"1min":0.67600000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.333,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.2090000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:30:03 2024","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.137,"15min":1.165},"min":{"1min":0.71099999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.49,"5min":1.79,"15min":1.79},"last":1.5069999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.085,"15min":1.135},"min":{"1min":0.67200000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.391,"5min":1.8660000000000001,"15min":1.8660000000000001},"last":1.478}]}]},{"osd":1,"up_from":15,"seq":64424509507,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":2871296,"data_stored":1948778,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.03,"5min":1.484,"15min":1.484},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86399999999999999,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.147}]},{"osd":2,"last update":"Fri Sep 13 09:30:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.40799999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.42599999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.97599999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.88500000000000001,"15min":0.85699999999999998},"min":{"1min":0.78800000000000003,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.3819999999999999,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":1.123},{"interface":"front","average":{"1min":1.089,"5min":0.95199999999999996,"15min":0.92900000000000005},"min":{"1min":0.57399999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.1859999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:30:50 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.242}]},{"osd":6,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.97899999999999998,"15min":1.069},"min":{"1min":0.70799999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.462,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.95899999999999996,"15min":1.0229999999999999},"min":{"1min":0.76100000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3480000000000001,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.214}]},{"osd":7,"last update":"Fri Sep 13 09:30:48 2024","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.0580000000000001,"15min":1.0589999999999999},"min":{"1min":0.70899999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.9099999999999999,"5min":2.048,"15min":2.048},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.0680000000000001,"15min":1.1399999999999999},"min":{"1min":0.68899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.819,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0589999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509506,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47800,"kb_used_data":2736,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324040,"statfs":{"total":96636764160,"available":96587816960,"internally_reserved":0,"allocated":2801664,"data_stored":1357599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72799999999999998,"15min":0.67100000000000004},"min":{"1min":0.49099999999999999,"5min":0.245,"15min":0.245},"max":{"1min":1.1950000000000001,"5min":12.441000000000001,"15min":12.441000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67500000000000004,"15min":0.71099999999999997},"min":{"1min":0.64100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0469999999999999,"5min":1.333,"15min":1.333},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:30:41 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:30:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.2050000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.796,"5min":12.555,"15min":12.555},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.84099999999999997},"min":{"1min":0.34499999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.9039999999999999,"5min":4.7439999999999998,"15min":4.7439999999999998},"last":0.93700000000000006}]},{"osd":4,"last update":"Fri Sep 13 09:30:53 2024","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.877,"15min":0.86099999999999999},"min":{"1min":0.76100000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2230000000000001,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.878},{"interface":"front","average":{"1min":1.0660000000000001,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.87,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.268,"5min":2.105,"15min":2.105},"last":1.1819999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:30:53 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":1.8260000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.96399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.091,"15min":1.095},"min":{"1min":0.70699999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.9670000000000001,"5min":12.519,"15min":12.519},"last":1.109},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.004,"15min":1.0620000000000001},"min":{"1min":0.77800000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.038}]},{"osd":7,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1779999999999999,"15min":1.1419999999999999},"min":{"1min":0.82099999999999995,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.0169999999999999,"5min":14.143000000000001,"15min":14.143000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.026,"15min":1.1020000000000001},"min":{"1min":0.87,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.169,"5min":2.169,"15min":2.169},"last":1.0840000000000001}]}]},{"osd":2,"up_from":207,"seq":889058230289,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":2732,"kb_used_omap":18,"kb_used_meta":28525,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":2797568,"data_stored":1357580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18654,"internal_metadata":29210402},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.91900000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.879}]},{"osd":4,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.82299999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:30:53 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.94899999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.097}]},{"osd":7,"last update":"Fri Sep 13 09:30:44 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.266}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11220,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8553,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24813,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14123,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:30:59.174 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T09:30:59.510 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T09:30:59.510 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T09:30:59.510 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T09:30:59.510 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:30:59.593 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T09:30:59.593 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T09:30:59.593 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:30:59.695 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T09:30:59.695 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T09:30:59.695 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:02.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T09:31:05.797 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T09:31:05.798 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T09:31:05.798 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:08.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T09:31:08.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T09:31:11.899 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T09:31:11.946 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T09:31:11.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:12.035 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T09:31:12.035 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T09:31:12.035 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:13.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T09:31:13.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T09:31:18.136 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T09:31:18.137 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T09:31:18.137 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:18.220 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T09:31:18.220 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T09:31:18.220 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:19.493 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T09:31:19.494 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T09:31:24.321 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T09:31:24.322 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T09:31:24.322 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T09:31:24.322 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:24.354 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T09:31:24.355 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T09:31:24.355 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:24.388 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T09:31:24.388 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T09:31:24.388 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T09:31:24.389 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:24.437 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T09:31:24.437 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T09:31:24.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:24.503 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T09:31:24.503 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T09:31:24.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:31:24.539 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T09:31:24.539 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T09:31:24.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T09:31:24.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-13T09:31:24.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-13T09:31:24.540 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T09:31:24.562 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:31:24.562 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T09:31:24.755 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:31:24.755 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T09:31:25.110 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:31:25.111 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T09:31:25.294 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:31:25.294 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T09:31:25.470 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:31:25.471 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T09:31:25.698 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:31:25.699 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T09:31:25.847 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:31:25.847 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T09:31:26.041 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:31:26.041 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T09:31:26.189 INFO:tasks.ceph:Archiving mon data... 2024-09-13T09:31:26.190 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/data/mon.a.tgz 2024-09-13T09:31:26.191 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-09-13T09:31:26.197 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.rBXhlVnuhQ 2024-09-13T09:31:26.198 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rBXhlVnuhQ 2024-09-13T09:31:26.327 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.rBXhlVnuhQ 2024-09-13T09:31:26.354 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.rBXhlVnuhQ is 518KB 2024-09-13T09:31:26.375 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.rBXhlVnuhQ 2024-09-13T09:31:26.382 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/data/mon.c.tgz 2024-09-13T09:31:26.382 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-09-13T09:31:26.430 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.EeuC5JpTqr 2024-09-13T09:31:26.431 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.EeuC5JpTqr 2024-09-13T09:31:26.606 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.EeuC5JpTqr 2024-09-13T09:31:26.631 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.EeuC5JpTqr is 705KB 2024-09-13T09:31:26.698 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.EeuC5JpTqr 2024-09-13T09:31:26.704 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/data/mon.b.tgz 2024-09-13T09:31:26.704 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-13T09:31:26.711 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.lZEtTeIHiB 2024-09-13T09:31:26.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.lZEtTeIHiB 2024-09-13T09:31:26.920 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.lZEtTeIHiB 2024-09-13T09:31:26.936 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.lZEtTeIHiB is 705KB 2024-09-13T09:31:27.004 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.lZEtTeIHiB 2024-09-13T09:31:27.010 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T09:31:27.010 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:31:27.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:31:27.028 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T09:31:27.028 DEBUG:teuthology.orchestra.run.smithi063:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:31:27.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:31:27.058 INFO:teuthology.orchestra.run.smithi063.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:31:27.102 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:31:27.109 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T09:31:27.111 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/remote/smithi063/crash 2024-09-13T09:31:27.111 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:31:27.129 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/remote/smithi099/crash 2024-09-13T09:31:27.130 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:31:27.147 INFO:tasks.ceph:Compressing logs... 2024-09-13T09:31:27.147 DEBUG:teuthology.orchestra.run.smithi063:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:31:27.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:31:29.541 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-13T09:31:29.541 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-13T09:31:44.763 INFO:tasks.ceph:Archiving logs... 2024-09-13T09:31:44.763 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/remote/smithi063/log 2024-09-13T09:31:44.764 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:31:46.274 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/remote/smithi099/log 2024-09-13T09:31:46.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:31:48.155 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T09:31:48.164 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T09:31:48.165 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:31:48.167 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:31:48.249 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T09:31:48.258 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T09:31:48.258 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:31:48.261 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:31:48.281 INFO:teuthology.orchestra.run.smithi099.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:31:48.281 INFO:teuthology.orchestra.run.smithi099.stdout:============================================================================== 2024-09-13T09:31:48.281 INFO:teuthology.orchestra.run.smithi099.stdout:+hv01.front.sepi 67.205.162.81 3 u 9 64 377 0.060 -0.187 0.028 2024-09-13T09:31:48.281 INFO:teuthology.orchestra.run.smithi099.stdout:*hv02.front.sepi 96.245.170.99 3 u 7 64 377 0.064 -0.043 0.044 2024-09-13T09:31:48.281 INFO:teuthology.orchestra.run.smithi099.stdout:+hv03.front.sepi 74.6.168.72 3 u 2 64 377 0.110 -0.021 0.025 2024-09-13T09:31:48.282 INFO:teuthology.orchestra.run.smithi099.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:31:48.285 INFO:teuthology.orchestra.run.smithi063.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:31:48.285 INFO:teuthology.orchestra.run.smithi063.stdout:============================================================================== 2024-09-13T09:31:48.285 INFO:teuthology.orchestra.run.smithi063.stdout:*hv01.front.sepi 67.205.162.81 3 u 7 64 377 0.080 0.546 0.931 2024-09-13T09:31:48.285 INFO:teuthology.orchestra.run.smithi063.stdout:+hv02.front.sepi 96.245.170.99 3 u 12 64 377 0.078 1.923 1.031 2024-09-13T09:31:48.285 INFO:teuthology.orchestra.run.smithi063.stdout:+hv03.front.sepi 74.6.168.72 3 u 13 64 377 0.086 0.647 0.983 2024-09-13T09:31:48.285 INFO:teuthology.orchestra.run.smithi063.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T09:31:48.286 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T09:31:48.295 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T09:31:48.295 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T09:31:48.314 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T09:31:48.330 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T09:31:48.347 INFO:teuthology.task.internal:Duration was 881.808050 seconds 2024-09-13T09:31:48.347 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T09:31:48.364 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T09:31:48.365 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:31:48.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:31:48.422 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T09:31:48.422 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-09-13T09:31:48.422 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:31:48.434 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-13T09:31:48.434 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:31:48.448 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T09:31:48.448 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:31:48.479 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:31:48.520 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T09:31:48.520 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:31:48.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:31:48.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T09:31:48.656 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T09:31:48.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:31:48.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:31:48.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T09:31:48.696 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:31:48.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:31:48.727 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-09-13T09:31:48.728 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-13T09:31:48.756 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:31:48.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:31:48.762 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:31:48.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:31:48.802 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T09:31:48.812 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T09:31:48.813 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/remote/smithi063 2024-09-13T09:31:48.813 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:31:48.844 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901036/remote/smithi099 2024-09-13T09:31:48.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:31:48.872 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T09:31:48.872 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:31:48.878 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:31:48.884 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T09:31:48.895 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T09:31:48.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T09:31:48.915 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T09:31:48.915 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:31:48.921 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:31:48.923 INFO:teuthology.orchestra.run.smithi063.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:31 /home/ubuntu/cephtest 2024-09-13T09:31:48.933 INFO:teuthology.orchestra.run.smithi099.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:31 /home/ubuntu/cephtest 2024-09-13T09:31:48.935 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T09:31:48.946 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T09:31:48.992 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/rbd_fsx_copy_on_read} duration: 881.8080499172211 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T09:31:48.993 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:31:49.069 INFO:teuthology.run:pass